Rails 24:00 Orvane Home & Outdoor

Rails architecture, in full

This is the long version of Rails in ten minutes: every part, what it owns, how the parts talk, and the choices behind them. Paths are relative to the repository root. Every figure quoted comes from the seed 23 hold-out run described in the README, with the command next to it there.

1. Overview

Rails has six parts and one rule: the part that thinks never holds the authority to act.

partpackageowns
worldrails/worldthe synthetic retailer, generated from a seed, and its answer key
dictionaryrails/dictionarythe definitions: SQL, owner, version, parameters, tests, change history
operatorrails/agentdetectors, cadence, runbooks, rules, the optional model, Controller, tools, scripted on-call people, replay
ledgerrails/ledgerthe append-only record, approvals, rejections, rollbacks, repeats
accuracyrails/recalmatching detections to the answer key; precision, recall, time to resolve, drift
screensappthe rails, play mode, ledger, approvals, accuracy, definitions, Ask, these documents

All state lives in one DuckDB file. The world tables, the ledger, the detector run times and the replay markers sit side by side, so a replay is a file copy and an audit is a SQL query.

2. The world (rails/world)

python -m rails.world.generate --seed N --out data/rails.duckdb --stats writes a complete retailer in about ten seconds, identical for a given seed. Seed 7 is used for development and tests, seed 11 for a second tuning pass, seed 23 is held out.

Times are naive local time in the world's one time zone. docs/world.md describes the tables and the planted kinds in detail.

3. The dictionary (rails/dictionary)

Each term is a YAML file in rails/dictionary/definitions/: a title, a description, parameters with defaults, and one or more variants. A variant has a key, an owner, a version, a change note, its SQL and its tests. Detectors are terms too: the SQL of oversell is the oversell detector.

4. The operator (rails/agent)

4.1 The pass

runner.step() is one pass at the clock's current logical time:

  1. housekeeping (a hook for timed states that end by themselves; no Rails action sets one, so it is empty),
  2. the detectors that are due by cadence (cadence.py: realtime every pass, hourly, daily at 08:00; last runs are kept in detector_runs so a restart does not repeat a daily check),
  3. items held back by the business-wide rate limit,
  4. new incidents, deduplicated against the 15-minute window and against incidents a person already rejected or rolled back (a person said no: the operator leaves that incident alone),
  5. proposals, in parallel, then decisions and actions one at a time in severity order.

Proposals may run on worker threads because they only think; decisions, tool calls and ledger writes stay on the runner's thread, in order, because budgets and cooldowns depend on what was just done.

4.2 Runbooks, rules and the model

policy.yaml gives each kind a runbook: the actions the operator may consider, in the order it tries them. A rule per kind (rules.py) fills in the arguments from the evidence. For oversell, duplicate orders and chargebacks the policy says rules decide. Elsewhere, when more than one action is possible, the optional model (llm.py, any OpenAI-compatible endpoint, configured only through the environment) is asked to choose and explain. A model answer that is not valid JSON, names an action outside the runbook, or misses an argument is discarded and the rule decides; the ledger records which happened. Proposals for the same kind, subject and evidence are reused within a run, so a repeated sighting never costs a second call. The seed 23 numbers in the README were produced with rules only.

4.3 The Controller

controller.py is the only code that can authorise an action. For every proposal it checks, in order:

  1. the action is known and not on the never list (unknown means tier 3),
  2. the action is in this kind's runbook,
  3. the arguments are complete, and the targets they name appear in the incident's evidence,
  4. the action's written condition holds (an evidence pack needs a delivery proof; a refund above EUR 100 needs a person),
  5. the action is outside its cooldown on this subject,
  6. budget: per subject per hour (4 automatic actions on one listing, order, lane or warehouse), per action per hour (30 listing pauses, 200 rerouted orders), and business-wide per minute (40 realtime actions; back-office kinds have their own limit and never use the realtime budget).

The outcome follows when_a_limit_is_hit: a used-up subject budget goes to a person, the business-wide limit waits for the next minute, a cooldown or a false condition is declined with the reason logged, an unknown action is never taken. Budgets are counted from the ledger, so they survive restarts and can be audited with a query.

4.4 Tools

Sixteen tools in rails/agent/tools/, each named after the call it stands for: resync_stock (PUT /listings/{id}/inventory), file_evidence_pack (POST /disputes/{id}/evidence), reroute_fulfilment (POST /fulfilment/orders/reassign), hold_order, resubmit_listing, update_tracking_eta, pause_listing, switch_lane_carrier, quarantine_batch, flag_account_review, request_carrier_intercept, refund_order, open_ticket, restore_price, block_account, accept_chargeback. Each one reads the rows it will touch, applies the change in the world tables, and returns the before and after snapshot with the call string. Undo is generic: it plans the compensating updates from the snapshot and applies them as new events, so history is never edited. A runbook step can chain a follow-up (hold_order then refund_order for a held duplicate); the follow-up goes through the Controller like any other action, so a refund above the limit still waits for a person.

4.5 Notifications and stakes

Tier 1 actions and tier 2 asks write a draft to the outbox for the people the policy names (the category manager, the logistics lead, the trading lead on call). Nothing is sent by Rails. stakes.py attaches what is at stake to every row, from the incident's own evidence: the money already hit or due and the orders affected. It never claims money "saved".

4.6 Scripted on-call people

The world is synthetic, so the people in it are too. humans.yaml defines a small shift (a trading lead on call, a trust analyst, payments, a logistics lead) with hours and response delays, and rules for how they answer: approve a duplicate refund, reject blocking an account whose returns are mostly resaleable, roll back a paused listing that was an approved loss leader. Their decisions land after their delay on the logical clock, as normal ledger approvals, rejections and rollbacks, each labelled "scripted on-call responses (synthetic)". Anything no rule covers stays in the queue for a person at the screen.

4.7 Process gaps

A kind with no runbook gives the operator no authority. It files one backlog row per kind in process_gaps and, when people have handled similar cases through the queue, drafts a runbook from what they approved, rejected and reverted. A person turns the draft into policy; the operator never does.

4.8 Shadow mode

With RAILS_SHADOW=1 the Controller decides as usual, but tier 0 and tier 1 actions are written as shadow rows saying what would have been done, and no tool runs. Tier 2 items still go to the queue.

5. The ledger (rails/ledger)

One table, ledger, append-only. Each row has: logical time, incident id, kind, subject, the proposal (source: rules or model, rationale, the model's own words when there is one), the Controller's decision and reason, the action and arguments, the tier, the result (call string, before and after, the undo function), the evidence, the definition term and version, model name and token counts, status, the person, a note, and the parent row.

Statuses: auto_done, shadow, escalated (waiting for a person), approved, rejected, rolled_back, declined, failed. Approving or rejecting an escalated row, or rolling back an executed one, writes a child row pointing at it; the original row is never changed. The approval queue is the set of escalated rows still without a decision. Repeats of the same problem inside the dedupe window go to ledger_repeats and are shown on the original row.

6. Replay and accuracy (rails/agent/replay.py, rails/recal)

scripts/replay_sale_evening.py refuses to run on data/ and expects a copy of the world. It replays the days before the sale at 5-minute steps (rules only, fast), then the sale evening at 30-second steps, with the scripted shift answering. Each part is marked in replay_events. With --save-start it saves a consistent copy of the database at the moment the sale evening opens: the start snapshot for live play. The run writes out/replay-<label>.json with the ledger summary, step timings and the accuracy figures.

Seed 23 wall times on a laptop: 70 to 80 seconds for six days of history (1,728 steps), 52 to 55 seconds for the evening (720 steps, 72 to 77 ms per step on average, under 190 ms at the slowest).

rails/recal/matching.py matches each detection to the answer key by kind, subject and time, with 30 minutes of slack either side. metrics.py computes precision (detections that match a planted fault), recall (planted faults matched), the share of incidents resolved by the operator's own action, reverted actions, time to resolve per kind (from the fault's start as its detector saw it to the resolving action), and which incidents rules or a model decided. drift.py watches the model's confidence, invalid answers and reverts over time. The Accuracy screen computes all of it from the ledger in the database it is pointed at, and names the weakest detector.

7. The screens (app)

A FastAPI application with Jinja templates and one small script, started with .venv/bin/uvicorn app.main:app --port 8167. RAILS_UI_SERVICES=real (the only mode) reads the database named by RAILS_DB, and the server refuses to start without it.

7.1 The rails

The main screen draws one frame of the evening at a logical moment, from /rails.json:

?at=HH:MM shows a fixed moment of the recorded run without touching the play clock.

7.2 Play mode

app/play.py keeps a logical clock over the evening that advances lazily at 10, 60 or 240 times wall speed. app/live.py runs the operator live: "From 18:00" copies the start snapshot to a working file, and a background thread with its own database connection steps the real operator on it (detectors, Controller, tools, ledger and the scripted shift) one 30-second tick at a time, as far as the play clock has gone. Every screen reads the working file while it runs, so approving an ask on the Approvals screen during play acts on the live run. The play clock is held to at most a minute ahead of the operator. "Recorded run to midnight" stops the live run and returns every screen to the replayed database. Without a start snapshot, play walks through the recorded run instead. On seed 23 a full live run of the evening writes the same 129 ledger rows by kind and status as the recorded run, in about 45 seconds of wall time (scripts/live_vs_recorded.py).

7.3 The other screens

Ledger (filters by kind, outcome and tier; each row opens with its evidence, chain, before-and-after and a rollback button where an undo exists), Approvals (the tier 2 queue with approve and reject, and the latest decisions by people), Accuracy, Definitions (with the tests runnable on the current world), Ask, and these documents rendered from docs/. A visitor's site arm (v, p query parameters) rides on every link, form and redirect.

8. Choices and their reasons

9. Plugging in a real retailer

  1. Map the order hub, channel connector, warehouse system, carrier feed and payments desk onto the tables in rails/world/schema.sql, through read-only replicas or change feeds.
  2. Run the dictionary's tests on the mapped data; fix the mappings until they pass.
  3. Run in shadow mode for a few weeks and score the "would do" rows against what people actually did.
  4. Point each tool at the real call it already names, with credentials scoped to exactly that call.
  5. Hand over tier 0 kind by kind, starting with the detectors whose precision held up in shadow mode.