Rails in ten minutes
Rails is an order operator for a retailer that sells on its own shop and on marketplaces. It watches orders, stock, listings, carriers, returns and disputes across the retailer's warehouses, finds what is going wrong while it is still cheap to fix, fixes what it is allowed to fix, asks a person for the rest, and writes down every step with the evidence behind it. This page explains how it is built and why. The full architecture goes module by module.
Everything shown here runs on a synthetic retailer, Orvane Home & Outdoor, generated from a seed: 40,000 products, three warehouses, the own shop plus two marketplaces (Marketvine and Tradewick), about 300,000 orders over 90 days, and a sale-day evening that takes eight times the orders of a normal Friday evening.
The shape of the problem
A multi-channel retailer lives on synchronisation. The same product is listed on three channels with three prices and three stock figures, fed from three warehouses through connectors that run every few minutes. On a sale evening the order rate goes up eightfold and every small drift becomes expensive: a listing that still shows stock the warehouse no longer has sells what cannot ship, a feed that drops a decimal sells a sofa below cost on one marketplace, a double click on a slow payment page charges a customer twice, a warehouse falls behind and the marketplaces' ship-by clocks keep running.
People handle all of this today, from dashboards and alerts, a few minutes or hours late. Rails does the routine part at machine speed and leaves the judgement calls to people, with the evidence laid out for them.
The loop
Every 30 seconds of logical time the operator runs one pass:
detectors ──► incidents ──► propose ──► Controller ──► tier 0/1: tool runs, ledger row
(dictionary (evidence (rules, (policy.yaml) tier 2: approval queue
SQL, due by rows) model only tier 3: declined, reason logged
cadence) if a choice)
- Detect. Ten detectors find ten kinds of problem: oversell, a listing a marketplace suppressed, a chargeback near its deadline with no evidence filed, return abuse on one account, a carrier delay cluster on one lane, a price below cost after a feed error, a duplicate order from a double submit, a warehouse about to miss the marketplaces' ship-by times, a delivery address changed to one the account never used, and a review bomb after a faulty delivery lot. Each detector is a definition in the dictionary: SQL with an owner, a version, parameters and tests. It returns rows, and each row becomes an incident carrying its evidence.
- Decide what is due. Not every check runs every pass.
policy.yamlsays which run in real time (stock, prices, orders, listings, lanes), which hourly (chargebacks, reviews) and which daily (return abuse, at 08:00). The same problem on the same thing is handled once per 15 minutes; later sightings are folded into the original ledger row as repeats instead of new rows. - Propose. For each incident the runbook in
policy.yamllists the actions the operator may consider, in order. Where the runbook leaves one sensible action, a fixed rule fills it in. A model is asked only where there is a real choice, and it is optional: without one, Rails runs on rules alone and every ledger row says so. - Check. The Controller is the only component that can authorise an action, and it is separate from whatever proposed it. It re-checks every proposal against the written policy: is the action known, is it in this kind's runbook, are its arguments complete and named by the evidence, is its condition true, is it inside its cooldown, and is there budget left (per listing, order, lane or warehouse per hour, per action per hour, and across the business per minute). It then applies the tier.
- Act. Tier 0 runs: push the warehouses' real stock to a listing, file a chargeback's evidence pack, move orders to another warehouse, hold an order. Tier 1 runs and tells the named people: pause a listing, switch a lane to its backup carrier, quarantine a lot, refund a held duplicate up to EUR 100. Tier 2 waits for a person: change a price, block an account, accept a chargeback, refund above EUR 100. Tier 3 never runs: exporting customer data, changing a marketplace's terms, contacting a customer about fraud, and anything the file does not list.
- Record. Each action is a tool named after the call it stands for (
PUT /listings/{id}/inventory,POST /orders/{id}/refund). It snapshots the rows it touches before and after, and the ledger row keeps that snapshot, so any action can be undone with a compensating change rather than by editing history.
Written authority, enforced in code
The policy is a YAML file a head of operations can read and sign: every action, its tier, its limits, who is told and how it is undone. Three properties matter.
- Anything not listed is never done. The Controller treats an unknown action as tier 3, and the
neverlist stays tier 3 even if someone adds one of its actions to the file by mistake. - Budgets are read from the ledger itself. The limit "at most 30 listings paused in an hour" is counted from ledger rows, so it survives restarts and anyone can audit it. When a single listing or order uses up its hourly budget, the next action on it goes to a person. When the business-wide per-minute limit is hit, the rest waits for the next minute.
- Conditions are part of the authority. A refund is tier 1 up to EUR 100 and asks a person above it; an evidence pack is filed only when a delivery proof exists. When a condition is false, the operator declines and logs why.
The ledger
The ledger is append-only. Each row carries the incident and its evidence rows, the definition and version the detector ran on, whether rules or a model decided, the call it made or would make, the tier, who approved, a plain-English reason, the before-and-after snapshot and how to undo it. Approvals, rejections and rollbacks are new rows that point at the row they answer, so the chain of one incident reads top to bottom. In shadow mode (RAILS_SHADOW=1) every tier 0 and tier 1 action is written as "would do" and nothing changes, which is how a new retailer starts.
Definitions shared by everyone
A retailer argues about words. "Late shipment" means one thing on a marketplace's scorecard (handed to the carrier after the ship-by time) and another to a customer (delivered after the promised date). "Active listing" means on sale to the marketplace team and sellable with stock to merchandising. Rails keeps one dictionary: oversell, late shipment, chargeback window, clean return, active listing, stock on hand, net sales, and the ten detector terms. Each has an owner, a version, SQL and tests, and the detectors, the operator, the screens and Ask all read it. Where a term has two valid meanings, both are kept, Ask returns both answers with how far they overlap, and nobody has to guess which number a dashboard meant.
Measuring itself
The generator writes an answer key, incidents_truth: every planted fault with its kind, time, subject and the correct handling, plus look-alike noise that should not be acted on. A replay runs the operator over a copy of the world on a logical clock and scores every detection against the key. On the hold-out seed 23, over the sale week (six days of history at 5-minute steps, then the sale evening at 30-second steps):
| seed 23 | |
|---|---|
| precision | 90% on 236 detections |
| recall | 97% of 213 planted problems |
| handled without a person | 83% |
| actions rolled back by people | 9 |
| weakest detector | changed delivery address, 73% precision |
The detectors were tuned on seeds 7 and 11 only, and every change is recorded in rails/dictionary/TUNING.md. The weakest detector is named on the Accuracy screen, not hidden: a changed delivery address looks like honest behaviour often enough that a share of its holds are false alarms, which is why it holds and never cancels.
The screens
- The rails. Orders move left to right on one track per sales channel through six stations: placed, paid, picked, shipped, delivered, returned. Each station shows how many orders sit there now and how many arrived in the last 15 minutes; each dot is one order from a fixed sample. Exceptions the operator handles are pulled off the line onto side rails, one per kind, along the evening's time axis, each with its receipt. Warehouse gauges show stock free to sell and the pick backlog against each warehouse's hourly capacity.
- Play. "From 18:00" copies the world as it stood when the sale evening opened and runs the real operator on it live, 30 seconds of logical time per step, at 10, 60 or 240 times wall speed. The receipts appear as it acts. A pass takes well under a second (72 to 77 ms on average in the seed 23 replays), so at 240 times the operator keeps up.
- Ledger, Approvals, Accuracy, Definitions, Ask. The record and its rollbacks, the tier 2 queue, the scores per kind, the dictionary with its tests, and questions answered from the dictionary's own SQL.
Plugging in
The detectors read tables, not systems. To run Rails on a real retailer, its order hub, channel connector, warehouse system, carrier feed and payments desk are mapped onto the tables the definitions read, through read-only replicas, and each tool is pointed at the real call it already names. It starts in shadow mode, so the early weeks of ledger are "would do" rows a person can score against what actually happened, before any authority is handed over.
The known gaps page lists what this demonstration does not prove.