Known gaps
What this demonstration does not prove, and where its numbers need care.
The world
- The world is synthetic. Every product, customer, order, warehouse, carrier and company in it is invented and generated from a seed. The figures measure the operator against problems the generator planted, with the look-alikes the generator planted next to them. A real retailer's noise is messier than any generator's.
- The world is fixed in advance. Orders, payments, picks, scans, returns and reviews are generated before the replay. When the operator acts (pushes real stock to a listing, pauses a listing, holds an order) the detectors see the change, but the orders the generator placed after that moment still exist: fixing a listing at 20:05 does not remove an order the world already placed on it at 20:07. The ledger shows what the operator did and when, not the sales it would have changed.
- Stock over the evening is reconstructed. The warehouse gauges show stock at a past moment as the stock at the world's end plus everything picked after that moment. The world has no inbound deliveries during the evening, so this is exact for the world; a real warehouse would also count goods received.
- Carriers collect in waves. On a normal day a van comes at 11:00, 15:00, 19:00 and 22:00 per warehouse (ten minutes apart); from 17:00 on the sale day every warehouse is collected every 40 minutes, staggered by 13 minutes, and the shipped station shows "next collection hh:mm" while no parcel has left in the last 15 minutes. Deliveries run 09:00 to 23:00; a parcel stays on the delivered station for six hours.
The operator
- Two detectors sit close to honest behaviour on purpose. A delivery address changed after payment to one the account never used is the weakest detector (73% precision on seed 23 over the week): honest customers do this. Return abuse is measured on a pattern over weeks (91% precision). Both act by holding or flagging, never by cancelling or blocking without a person.
- Prices below cost are the weakest in the evening alone (68%). The world plants approved loss leaders whose approval is recorded on the own-shop listing only; the operator pauses the marketplace copies and the trading lead puts all eight back on sale. The rollbacks are counted against the operator.
- Review bombs are found late or not at all when they are small. The hourly detector needs at least five reviews of two stars or fewer on one product within its lookback; on seed 23 it found 2 of the 4 planted bombs.
- Evening recall reads low (65%). Problems the operator handled in the days before (an abusive account, a chargeback answered days earlier) are still open in the answer key during the evening. The week figure (97%) counts each problem once.
- The seed 23 figures were produced with rules only. The optional model path (an OpenAI-compatible endpoint asked only where a runbook offers a real choice) is built and logged, but no figures here come from it.
- Time to resolve for return abuse is weeks. The daily check sees a pattern only once enough returns have come back, so the time from the pattern's start to the flag is long by construction.
Actions and people
- No call leaves the building. Each tool changes the world tables and records the call it stands for (
PUT /listings/{id}/inventory); no marketplace, warehouse, carrier or payments system is called. Messages to the people named in the policy are drafts in an outbox. - The people answering the queue are scripted. A synthetic on-call shift approves, rejects and rolls back by rules in
rails/agent/humans.yaml, after a delay on the logical clock. Every row they write is labelled as scripted. Their choices shape the "handled without a person" and "rolled back" figures.
The screens
- One clock for every visitor. Play mode runs in the web server's process with one shared clock and one live run. "From 18:00" restarts the live run for everyone looking at it, and a deployment with several server processes would give each its own clock.
- The live run is the operator, not a recording, so it could drift from the recorded run. It starts from a snapshot of the database taken when the sale evening opens, and decisions the scripted shift had scheduled but not applied before 18:00 are not carried into it (the shift's backlog sweep picks those items up). On seed 23 the two agree: a full live run of the evening wrote the same 129 ledger rows by kind and status as the recorded run, in 45 s of wall time (
.venv/bin/python scripts/live_vs_recorded.py). A different seed or a model in the loop could differ. - Decisions on the recorded run are kept. Approving, rejecting or rolling back on the recorded run (outside play) writes to the replayed database, as it would in production. Copy the world and replay again to start clean.
- Dots are a sample. Each dot on the rails is one order out of every 23 (by order id). Station counts are exact.