Agent docs

Human in the loop

> Evidence: 0 file:line witnesses as of 2026-08-18.

Evidence: 0 file:line witnesses as of 2026-08-18. Confidence: derived, written from the seams and the estate's failures rather than from citations; treat each rule as a design call and argue with it. What changes it: the first module built against this doc (../PLAN.md forge-1) and its instance breakdown.

How a module asks for a person, how the app holds the item, and how the resolution comes back. The business rule this serves, in Christina's words: "when the agent's confidence is below the bar, it does not guess and does not silently drop. It parks the SKU in QC, tagged with the stage it got stuck at, and keeps everything else moving." The contract's job is to make that true across a module boundary without the module owning a queue.

Audience: anyone building a payload that can be unsure, or an app with a review surface.

What the instances do#

versable-runnerspeedwaywalmart-mvp
how a payload signals doubtit does not; an unverifiable item is a non-retryable error (RESEARCH_QUALITY_LOW)items land in a review queue; "review-quiet" is an orchestrator trigger; consent gates the next stagePartError rows, one per (part, field), are the queue; resolve or skip re-runs evaluate_job
where the queue livesnowherein the app (Firestore, manual-review-queue-plan.md)in the app (Postgres)
how resolution re-entersn/athe next stage runs after the queue is quiet or a human says proceedfinalize_resolve clears the error and re-evaluates the job
bulk resolutionn/ain the review UIError management, grouped by field

versable-runner turning "I cannot verify this" into an error is the lapse: the caller cannot tell "the item is broken" from "a person should look". The two apps have the queue in the right place and neither can express it to a module.

The rule#

A module emits needs_review as an item outcome; the app owns the queue, the human, and the bar; resolution comes back as a new job.

The module's half#

  • A payload that cannot settle an item without a person returns needs_review with a stable reason from the manifest, its confidence and judged_by, its evidence (what it looked at, what it found, why it is unsure), and its best partial output if it has one (03-jobs-and-state.md). It does not guess and it does not error.
  • needs_review is terminal for the module. The job completes as completed_with_review (or completed_with_errors if both exist) and the runner moves on. Nothing in the module waits for a person.
  • The review outcomes are listed on their own route (GET /jobs/{job_id}/review), paginated, grouped by reason on request, because that is what a review UI wants first.
  • Every result also carries confidence and judged_by, so an app can hold an item the module called done. The module's threshold for emitting needs_review is the floor ("I genuinely cannot"), the customer's bar is the app's, and the app's is usually higher.

The app's half#

  • The queue: held items, tagged with the stage (capability) and reason, grouped, bulk-resolvable, with the evidence shown. This is the SKU state machine's held state and it lives with the SKU, in the app.
  • The bar: per customer, per capability, tunable; applied over module confidence and any judge the app ran itself (14-graceful-degradation.md, judges are capabilities).
  • The human's verdict: resolve with a value, waive ("move on anyway"), reject, or ask the module again with more input.
  • The teaching loop: a resolution that should never recur becomes a rule or a config value the app passes on the next job. Christina's note calls this the compounding asset; it is entirely app-side, and the module's contribution is that its outcomes carry enough evidence to learn from.

Resolution comes back as a new job#

A resolved or waived item that needs the module again (re-generate with the human's correction, re-match with the human's hint) is submitted as a new job with parent_job_id, one or a few items, the resolution carried in the item payload or params. The module treats it as any job. This keeps every job immutable, keeps the module stateless with respect to the review, and means "re-run this one item" is the same call as everything else. It is also the debug path (07-observability.md, log_level: debug on a single-item child job).

An app that resolves an item without needing the module again (a human typed the value) does not call the module at all; the app's store is the truth for the SKU.

What "keeps everything else moving" means at the module#

Items that need review do not block items that do not. A job with 30 review items and 4,970 results completes; the app holds 30 SKUs and advances 4,970. Nothing in a module waits, batches, or gates on a review, which is why the review queue cannot live there.

Cost of review is a meter#

A needs_review outcome records the usage that produced it like any other; the app decides whether held items count against a plan. A judge that ran records its own events under its own capability. Per-reason counts in /jobs/{job_id}/stats are how the "QC volume decays every month" claim gets measured.

Review is a standard optional step, not a capability's private business#

Owner ruling, 2026-08-23, with the three instances as its evidence. Review is a first-class mechanism every capability can use, declared the same way and consumed the same way, and it must be possible to turn it off entirely.

What each instance got wrong, in the owner's words. enhancement-product's "whole issue was the review and validation had to be tagged as a modification, so we'd try to fail the row or try to cram it directly into a capability". speedway "had review support from the get go but it was very crude and just for one customer". walmart "just didn't get it built properly". Three different failures, one root cause: review was never its own step, so each app either disguised it as something else or hard-wired it to one customer.

The shape. Review is a standard step in the run, offered by the contract rather than invented per capability:

  • Every capability may declare a review step. The declaration is uniform, so a consumer renders one queue across all four rather than four bespoke surfaces.
  • The step's criteria are pre-encoded in the contract for now. Customer-specific criteria arrive later as code, rules, or config, and the shape must leave room for that without a schema change.
  • The step can be a no-op, and this is a first-class outcome rather than an omission. A configured no-op runs, raises no objection, and reports all-ok, or exits early with the same result. A run with review disabled is a run whose review step succeeded trivially, not a run missing a step.
  • A review outcome is never a modification of the item and never a failure of the row. It is its own verdict alongside the work, which is the specific trap enhancement-product fell into.

Why the no-op matters more than it looks. A customer who wants no human gate, a capability whose confidence is always sufficient, and a pipeline being smoke-tested all need the same thing: the step present, declared, and silent. If disabling review means removing the step, then the configured-off path and the not-implemented path are indistinguishable to a consumer, and the first time someone needs to know which one they are looking at is an incident.

The posture this leaves open, deliberately. Whether an unsure row parks or proceeds with a default is then a per-capability, per-customer setting rather than a property of the product. walmart's matcher proceeds with a catch-all type; speedway holds for review. Both become configurations of one mechanism instead of two different products.

Do-nots#

  • Do not turn "I cannot verify this" into an error. Emit needs_review. (versable-runner RESEARCH_QUALITY_LOW)
  • Do not hold, wait on, or gate a job in a module because an item needs a person.
  • Do not build a review queue or a threshold policy inside a module.
  • Do not mutate a job to record a resolution. New job, parent_job_id.
  • Do not emit needs_review without a stable reason and evidence; a reasonless hold is a guess with extra steps.
  • Do not implement review inside a capability, or as a modification of the item. It is its own declared step with its own verdict.
  • Do not build a review surface for one customer. The declaration is uniform or the consumer cannot render one queue.
  • Do not express "no review" by removing the step. Configure it to a no-op, so a consumer can tell a disabled gate from an unbuilt one.
  • Do not fail a row because it needs a person. Parking and failing are different outcomes and a consumer acts on them differently.
@versable-git/ui · reference, canon, and method, read in place