I wanted to write down what actually happens between a player tapping an answer and four other screens updating, both ways we can do it, with nothing hand-waved. The sources are the three repositories themselves, read on 2026-09-18, plus the shared log two coding agents kept while they built the new path overnight. Both transports, then what happens to the points, including the parts that are not finished.
Hearso is standup icebreaker trivia: a host opens a room, teammates join with a short code, and ten ten-second questions run on a server-authoritative clock. Scores from a finished game land on a shared leaderboard in its own AWS account.
Say the status up front. Every room in production is driven by an HTTP poll every 1.5 seconds. The WebSocket service (hearso-rt, Rust) and the browser client that speaks to it were built and run on one laptop on 2026-09-18, behind a per-room switch called rooms.transport, stamped once at room creation and poll unless a deployment explicitly says otherwise. None of it is deployed: the migration that adds the column has not reached a hosted database, and the web work sits on a local branch 45 commits ahead of develop. Everything here about the socket is true of a laptop; everything about polling is true of hearso.com.
| Word | What it means here |
|---|---|
| room | One shared space, opened by a host. It moves lobby → question → reveal → finished exactly once. |
| host | The player who opened the room. Starts it; the game advances itself from there. |
| round | One question. Ten rounds make a game. |
| snapshot | The whole room as one player should see it now: the same JSON the poll returns. |
| ticket | A 30-second, single-use credential that lets one browser open one socket. |
| outbox | An ordered table of finished games, written inside the finish transaction. |
| transport | Which service drives a live room: poll (Next.js) or ws (hearso-rt). |
Figure 1 is the whole system on one page; the rest of the post walks its arrows in order.
Figure 1. The pieces, read from the source on 2026-09-18. Solid lines run in production; dashed lines ran on one laptop that night.
The poll, which is still how it works
A player taps an answer. The browser sends POST /api/rooms/[code]/answer with a choice index. The route reads the room, refuses if no question is open, then makes one conditional write: it adds this round's answer to the player's answers object and raises their score, but only WHERE NOT (answers @> '{"1": {}}'). Only while this round's key is still absent. Two taps race, one wins, the other gets the winner's receipt.
Timing is never the browser's business. The room row carries question_started_at, stamped server-side, and the route computes the elapsed milliseconds itself. A correct answer is worth 100 + round(100 × (10,000 − elapsed) / 10,000), so 900 ms earns 191 and a 9-second answer earns 110.
Everyone else finds out on their next poll. GET /api/rooms/[code]/state runs every 1.5 seconds from every open page, and it is not only a read: it is the game loop. A poll that finds a question older than its 10-second limit plus a 2-second grace flips the room to reveal; one that finds a stale reveal advances it, or finishes the game after the last round. Those are conditional updates too, so polls arriving at once collapse into one transition.
That is a genuinely good first version and I would build it again. No connection lifecycle, no second process to watch, and nothing that can wedge a room, because there is no single driver to lose.
Why bother with a socket
Two costs, and only one is latency.
The first is staleness you cannot design away. A change lands in Postgres at some instant; every other screen learns about it on its next poll, up to 1,500 ms later and 750 ms on average. Fine for a roster; not fine for a reveal, which is the moment the whole room is supposed to share.
The second is traffic that scales with people rather than events. At a 1.5-second interval each player makes 40 requests a minute (60,000 / 1,500), whatever is happening. On a socket the keep-alive is one ping every 10 seconds: 6 a minute per player.
Figure 2. Arithmetic, not a measurement: 60,000 / 1,500 = 40 and 60,000 / 10,000 = 6 per player per minute, from `POLL_MS` in the room page and `heartbeatMs` in the welcome frame. It counts the keep-current traffic only.
Both numbers in Figure 2 are exact derivations from two constants in the source, not a benchmark. In production there is no socket to benchmark.
The ticket: a cloakroom token for a service that cannot read your cookie
Think of a cloakroom. You hand over your coat and get a numbered token; the counter never needs your name. The token is the whole claim, worth nothing to anybody else, spent the moment it is used.
Precisely: a ticket is a short-lived bearer credential signed by one service and verified by another with no callback. Ours is v1.<payload>.<signature>, the signature HMAC-SHA256 over "v1." + payload, the payload compact JSON with its keys in a frozen order: {"v":1,"jti":…,"sub":…,"room":…,"aud":"hearso-rt","iat":…,"exp":…}. It lives 30 seconds (exp = iat + 30000), the verifier forgives 5 seconds of clock skew, and the first use burns its jti.
We needed one because the room actor cookie is httpOnly and path-scoped to /api/rooms/<code> on our own origin, so a handshake to another origin carries none of it. Next.js can read that cookie; the Rust service cannot. So the browser asks Next.js, which checks the cookie, that the room is ws, and that the seat is still held (left_at IS NULL), and presents the answer in the socket's address.
Figure 3. One connection attempt, read from the source on 2026-09-18. `K7M2Q9XP` is the room code in our golden fixtures, not a real room.
The burn in Figure 3 is one Valkey command: SET ticket:<jti> 1 NX PX 40001. NX only succeeds if the key is absent, so a second use gets nil and is refused with AUTH_TICKET_REPLAYED. Both sides pin the same frozen test vectors, so a change to either implementation that would break the other fails a test rather than a game.
One round, over the socket
A ws room has exactly one writer. Each room gets an owner task: a tokio task with a bounded mailbox (capacity 32) that every command and every timer for that room passes through, one at a time. There is no cross-task locking because there is nothing to lock against.
The answer command is {"v":1,"id":"<uuid>","type":"room.answer","data":{"index":1,"choice":1}}, at most 4,096 bytes. index is there for one reason: a frame can arrive after the room has moved on, and naming the round is what lets the service refuse a stale answer instead of scoring it against the next question.
Figure 4. One round of a four-player game, read from the source on 2026-09-18.
The database write, third in Figure 4, is the route's shape exactly: a SELECT … FOR UPDATE on the room at the expected index, then an UPDATE guarded on the round key still being absent. The owner task then rebuilds the room for every viewer and publishes it, and that publication is the part I like most.
A snapshot travels over a tokio::sync::watch channel, which holds exactly one value: the latest. Think of a whiteboard rather than an inbox: the writer overwrites it, and a reader who looks away and comes back sees the current state, not a queue of what they missed. A page busy rendering during three rapid changes receives one frame carrying the newest room; the two it missed are gone. The client also drops anything whose version is not greater than what is on screen, so a room can never walk backwards. That is how memory stays bounded when a phone throttles a background tab.
The strangler: the same JSON on both pipes
The rendering code did not change, and that was the design.
room.snapshot carries the exact body GET /api/rooms/[code]/state returns for that viewer, with one key added: "transport":"ws". Commands mirror the seven POST routes, and a cmd.result carries the status and body that route would have returned. So the page has one function that either does a fetch or sends a command, and every call site reads ok, status and json() as before. A reply that never comes is status 0, which is what a failed fetch already was.
The handover is one branch: the first poll of a ws room comes back saying "transport":"ws", the page opens a socket, the interval stops. A polled room's payload has no such key.
A room never changes transport, and nothing updates the column after the insert: that is what lets both sides assume a single writer for a room's whole life. Every reader treats an absent column, a null and an unrecognised value as poll, because on a database the migration has not reached, nothing could have stamped a ws room.
Finishing a game exactly once
When a ws game ends, hearso-rt runs one Postgres transaction: the conditional UPDATE … SET status='finished' that decides the race, a games ledger row, a player_profiles bump per player, one plays row per ranked player, and one row in game_finished_outbox.
An outbox is a table written inside the same transaction as the thing it describes, so the record and its announcement cannot disagree. Ours holds {"v":1,"roomCode","gameRound","finishedAt","plays":[{"playerId","rank"}]}, keyed by an always-growing seq, unique on (room_code, game_round, finished_at). Badges stay in Next.js, because the achievement rules live there, and writing them again in another language in another repository is how two writers come to disagree about what a badge means.
After the commit, the service calls POST /api/internal/games/finished with {"outboxSeq":N}, signed v1=<hex HMAC-SHA256(secret, "<unix ms>.<raw body>")> and refused outside 60 seconds either way. The route drains every pending row up to that seq in ascending order: award first, then mark, the mark conditional on the row still being unclaimed. A row it cannot handle stops the drain, and the answer reports an offset below it. That is the difference between a committed offset and a queue that leaves a hole nobody notices until somebody asks why one player's first game was never recorded.
The night it was first wired end to end, the callback drained two finishes in 0.12 s and 0.06 s, and the same call swept up two older rows, 741.85 s and 915.10 s after their finishes (cutover log, web-side entry, 2026-09-18T11:57:48Z).
Where the points go
Only a server submits a score. The leaderboard service has no player authentication by design (its trust model is that the calling game vouches for its players) so the deployment boundary is the trust boundary. lib/leaderboard-client.ts throws at module load if it finds itself in a browser.
A finished poll game sends one POST /v1/scores per player with an Idempotency-Key of trivia-<roomCode>-<playerId>, so a retry is safe. API Gateway hands it to one Lambda: nodejs22.x on arm64, 256 MB, a 10-second timeout. That Lambda writes three boards by default (all-time, the achievedAt day and its ISO week) computed from when the score was achieved rather than when it arrived, so a score set at 23:59:58 lands on that day's board. We ask for the day's board only when the player is a guest, to keep write costs down.
Each board is a DynamoDB item keyed playerId (partition) and leaderboardId (sort), where leaderboardId is <gameId>#<variant>#<period>: say trivia#alltopics#daily-2026-09-18. The write is conditional on attribute_not_exists(#score) OR #score < :score; a submission that beats nothing moves lastPlayedAt and nothing else. Both tables bill per request, and a TTL on expiresAt retires daily rows 35 days past their period end and weekly and monthly rows after about 400. Nothing streams off the table: there is no consumer downstream of the write.
The interesting key is the other one. Imagine a library where, instead of sorting the shelf when somebody asks, you choose each book's spine label so that shelving it alphabetically is the ranking. That is rankKey: <invertedScore>#<achievedAtMillis>#<playerId>, each number zero-padded to 13 digits, the score stored as 1,000,000,000,000 − score. A score of 191 achieved at epoch millisecond 1789700000000 becomes 0999999999809#1789700000000#<playerId>. A higher score gives a smaller string, and so does an earlier timestamp, so one ascending read of the RankIndex global secondary index answers "highest first, earliest wins ties" with no comparison logic downstream.
Figure 5. Propagation from `POST /v1/scores` to the score being readable: p50 159 ms, p95 210 ms, p99 322 ms over 300 samples, DEV stack in `us-east-1`, run started 2026-08-18T19:38:19Z, error rate 0.00%. Per board size: 100 players 160/235/449 ms, 1,000 players 158/181/206 ms, 10,000 players 158/210/265 ms. The sizes ran one after another, so this is not a controlled comparison between them (`Lapis-Foundry-Labs/online-leaderboard#49`).
Worth setting against that: the very first score we ever wrote to real AWS took 3,265 ms on a cold-start sample (Lapis-Foundry-Labs/online-leaderboard#30), about twenty times the warm median. A benchmark measures a warm system, and the first request of the day is not one.
What is not done
- Nothing is deployed. No VPS, no TLS, no DNS. The Rust service has only ever been reached over loopback, under a bot swarm on one laptop rather than traffic.
- A
wsgame's scores never reach the leaderboard. The Rust service writesplays, writes the outbox row and calls the callback, but has no score submission at all, andsubmittedstays false on every seat of awsroom (cutover log, service-side entry, 2026-09-18T11:55:00Z: "Leaderboard target planning/submission remains unimplemented in the Rust crate"). - The chaos harness is a stub.
crates/simis 43 lines; none of the eight invariants the plan asks it to check exists, so its seed count is scaffold evidence and nothing more. The service's own scaffold pull request said as much from the start: its greensimresult "only proves the empty crate builds" (Lapis-Foundry-Labs/hearso-ws-game-service#1). - The ticket is burned before the room and seat are checked. Present a valid ticket to a polled room and it is spent on a connection that is then refused 409. Harmless, since the browser mints another, but it is the wrong order.
- One leaderboard game id per process.
gameId()reads the deployment tier, so everything a deployment submits goes to the same board; per-game ids are not built. The leaderboard's own production stack does not exist either: its README says so, and Figure 5 is from the DEV stack. - The two-hour soak had not finished when this was written. At minute 90 of 120 its live counters read 260 games finished, 404 reconnects, 133 leaves, zero protocol violations, zero unexpected closes, zero 5xx, snapshot p95 1 ms, memory 1.56% below baseline. Readings, not a verdict, so there is no soak result here (cutover log, web-side entry, 2026-09-18T21:24:25Z).
Key takeaways
- A 1.5-second poll is a real transport, not a placeholder, and it is what production still runs. Its price is not only latency (750 ms on average) but traffic that grows with people rather than events.
- Pushing the same JSON over the new pipe is what let the rendering code stay untouched. The call sites changed by one word.
- A room that can never change transport lets both services assume a single writer, and lets every unknown value fail safely to
poll. - Latest-wins delivery over a
watchchannel means a slow client skips snapshots instead of queueing them. That is how memory stays bounded. - Write the announcement inside the transaction it announces, then drain it as an ordered log with a committed offset. Stopping at the first bad row is what keeps that offset honest.
What we would do next
Put the leaderboard submission into the Rust finish, since a ws game is currently the only kind whose points go nowhere. Build the chaos harness for real, with its eight invariants, before anything reaches a server: a green result that only proves an empty crate compiles is a check whose null case reads as success, and that shape has caught us before. Then deploy behind TLS and turn exactly one room ws.
Two smaller things after that: move the ticket burn to after the room and seat checks, and give the web client a leaderboard game id per game rather than one per process.
Evidence
All times UTC. Paths are relative to each repository.
| Claim | Source |
|---|---|
Poll interval 1,500 ms; the handover branch on transport: "ws" | hearso-web app/room/[roomId]/room-client.tsx (POLL_MS, openSocket) |
| Answer scoring, 10 s limit, 2 s grace; 900 ms → 191 points | hearso-web lib/game.ts; lib/fixtures/state/question-board-on.json |
| Conditional answer write; the state route as the game loop | hearso-web app/api/rooms/[code]/answer/route.ts, app/api/rooms/[code]/state/route.ts |
rooms.transport stamped at creation; absent, null and unknown read as poll | hearso-web lib/room-transport.ts, lib/room-transport-switch.ts, app/api/rooms/route.ts |
| Ticket format, 30 s life, 5 s skew, frozen key order | hearso-web lib/ws-ticket.ts; cutover/WS-CUTOVER-PLAN_1.md C2 |
Ticket route: cookie only, ws rooms only, live seat only, no-store | hearso-web app/api/rooms/[code]/ws-ticket/route.ts |
| Origin → verify → burn → 409 → 403, all before the upgrade; 4,096-byte frames; 4009 on replacement | hearso-rt crates/transport-ws/src/lib.rs |
SET ticket:<jti> 1 NX PX 40001; AUTH_TICKET_REPLAYED | hearso-rt crates/auth/src/lib.rs |
heartbeatMs: 10000 in session.welcome; last 64 command ids replayed | hearso-rt crates/transport-ws/src/connection.rs; hearso-web lib/room-socket.ts |
| One owner task per room, mailbox capacity 32; snapshots published per viewer | hearso-rt crates/bin-server/src/main.rs; crates/transport-ws/src/lib.rs |
Latest-wins delivery (watch + borrow_and_update); versions must increase | hearso-rt crates/transport-ws/src/wire.rs; hearso-web lib/room-socket.ts |
Answer command: {index, choice}, stale index refused 409, FOR UPDATE + conditional UPDATE | hearso-rt crates/rooms/src/commands.rs, crates/persistence/src/answers.rs |
Snapshot is the state route's payload plus "transport":"ws" | hearso-rt crates/rooms/src/snapshot.rs; hearso-web lib/fixtures/state/*.json; plan D5 |
Command routes answer 409 room_uses_realtime for a ws room | hearso-web app/api/rooms/[code]/{start,answer,advance}/route.ts |
Finish transaction: status CAS, ledger, profiles, plays, one outbox row | hearso-rt crates/persistence/src/finish.rs |
Outbox schema, seq identity, unique (room_code, game_round, finished_at), 7-day prune | hearso-web supabase/migrations/20260918010100_add_websocket_room_transport.sql |
Signed callback: v1=<hex HMAC-SHA256>, 60 s either way, retries at 1 s / 4 s / 10 s | hearso-web lib/internal-callback.ts, app/api/internal/games/finished/route.ts; hearso-rt crates/leaderboard/src/lib.rs |
| Drain in seq order, award then mark, stop at the first failure | hearso-web lib/finish-outbox.ts |
| Drains of 0.12 s and 0.06 s; sweep of two older rows at 741.85 s and 915.10 s | cutover log, web-side entry, 2026-09-18T11:57:48Z |
| Server-only submission; the trust boundary is the deployment boundary | hearso-web ARCHITECTURE.md §3, docs/agent-brief.md §2, lib/leaderboard-client.ts |
Idempotency key trivia-<roundId>-<playerId>; default fan-out all-time + daily + weekly; guests scoped to ["daily"] | hearso-web lib/leaderboard-targets.ts, lib/profile.ts (GUEST_BOARDS); online-leaderboard src/lib/score-update-command-builder.js |
Lambda nodejs22.x, arm64, 256 MB, 10 s; PAY_PER_REQUEST; TTL on expiresAt | online-leaderboard template.yaml |
Table keys, RankIndex key schema and INCLUDE projection | online-leaderboard template.yaml; docs/adr/0010-table-and-index-design.md |
rankKey shape, SCORE_CEILING 1,000,000,000,000, PAD_WIDTH 13; conditional improvement write | online-leaderboard src/lib/score-key-builder.js, src/constants/score-limits.js, src/lib/score-update-command-builder.js |
leaderboardId = <gameId>#<variant>#<period> | online-leaderboard src/lib/leaderboard-id-parser.js |
| Retention 35 / 400 / 400 days; all-time never expires | online-leaderboard src/constants/leaderboard-retention.js |
Propagation p50 159 / p95 210 / p99 322 ms, 300 samples, DEV us-east-1, 0.00% errors | online-leaderboard benchmarks/20260818T193819.881Z/report.md; Lapis-Foundry-Labs/online-leaderboard#49 |
| First live write 3,265 ms on a cold-start sample | Lapis-Foundry-Labs/online-leaderboard#30 |
| Production leaderboard stack not deployed | online-leaderboard README.md, Environments |
| Nothing deployed; VPS, TLS and hosted migrations out of scope | cutover/WS-CUTOVER-PLAN_1.md §1 |
No leaderboard submission in the Rust service; submitted false on ws seats | hearso-rt crates/bin-server/src/main.rs (no hearso_leaderboard submission call); cutover log, service-side entry 2026-09-18T11:55:00Z and web-side entry 2026-09-18T11:57:48Z |
crates/sim is 43 lines and checks none of the eight invariants | hearso-rt crates/sim/src/lib.rs; plan §8; Lapis-Foundry-Labs/hearso-ws-game-service#1 |
| Soak unfinished; minute-90 live counters | cutover log, web-side entry, 2026-09-18T21:24:25Z |
Branch 45 commits ahead of develop | git log --oneline origin/develop..HEAD, read 2026-09-18 |