The doc the owner says keeps getting rewritten in every app: how an agentic
capability (a model in the loop: enhancement copy, image generation,
attribute normalization, part-type matching) is set up, structured,
interfaced, and layered, and what the options are at each layer. It sits
beside 00-building-a-module.md (which covers any module) and adds what is
specific to payloads that call models. It is a recipe book, on purpose: the
owner's stance is that "we cannot delude ourselves by saying that we will
build the end-of-history FINAL version of each"; so where the four V1 modules
diverge, the divergence is written as a recipe with its ingredients showable,
not smoothed into a standard.
Vocabulary naming test two: everything here is a payload on the
runner, with fittings reached through Context, reference data
for taxonomies and prompts that are published, and the divergent bits
labelled bespoke or filed as recipes. Two words fit without strain; one
friction, noted below: prompts sit between "reference data" and "system
config" and the words do not pick for you.
Audience: an agent or a person writing enhancement, image generation, attribute normalization, or part-type matching as a module, and whoever reviews it.
What is the same every time, so it is factored#
The owner's observation is that "SO much of it is still the same and well quantifiable". This is that list, and every line is something the runner or a fitting provides, so the payload never writes it:
- one item in, one outcome out, retries owned by the runner (
canon/03,canon/04) - every model call traced (
ctx.trace), rate-limited and breakered (ctx.limit,ctx.guarded), metered by tokens and cost (ctx.meter,contracts/usage-event.md), and cached by input when the manifest says (ctx.cached) - confidence and evidence on every result, and
needs_reviewwith a stable reason instead of a guess (canon/10) - reference data loaded as a versioned snapshot, version stamped on the
outcome (
canon/05) - a per-item log a caller reads without a shell (
canon/07) - a manifest that names params, item, output, codes with labels, variants,
meters, and outputs (
contracts/manifest.md)
If a payload finds itself writing any of these, stop: it is a runner gap,
add the verb (contracts/runner-verbs.md).
The layers of an agentic payload#
Every one of the four modules has these layers; what differs is the recipe in each. Name the layer, then pick or record the recipe.
item ──► prepare ──► retrieve ──► generate ──► self-check ──► judge ──► settle ──► outcome │ │ │ │ │ │ normalize reference model call rules, second result | input, data, web, with prompt schema, opinion needs_review | templates cache + params confidence (variant) error| Layer | What it does | Options seen in the estate (recipes) | Where it lives |
|---|---|---|---|
| prepare | turn the item and params into what the model needs: template substitution (<<Column>>), field selection, normalization of the part number | App V5 template DSL with <<llm:topic=…>> spans; speedway's frozen content template per job; walmart's spec-sheet mapping | payload; templates are per-tenant config passed in params |
| retrieve | what the model is allowed to know: reference data (taxonomy, PCdb, accepted values), web research, prior outcomes | App V5 research mode via Vertex grounding; speedway scrape then normalize as two runs; walmart scrape stage feeding content | payload behind ctx.reference, ctx.cached; research as a variant (self or extractor) |
| generate | the model call | Gemini via Vertex (speedway, walmart); OpenAI and Gemini for images (App V5); model chosen per variant | payload, always through ctx.trace and ctx.guarded; model id in the manifest's variant options |
| self-check | static rules and a self-reported confidence: length limits, banned characters, must start with the part type, schema validity, "unverifiable claim" | App V5's RESEARCH_QUALITY_LOW; the owner's draft's Goal classes (satisfactory, needs review, retry, failed) | payload; static checks are code, confidence is judged_by: self |
| judge | a second opinion when self-report is not enough | a chained judge capability inside the module, a standalone judge module, or the app's own | a variant; ctx.judge; the manifest says which score is authoritative (adr/004) |
| settle | decide the outcome: result with confidence, needs_review with reason and evidence, or terminal error | net-new attribute values held for review (attribute normalization); ambiguous part type held with candidates (matching); low-confidence copy held with the draft | payload returns the outcome; the runner records attempts, usage, timings |
Options and recipes, by module#
The four V1 modules, each with what it shares and what is its own. Each "own" line is a recipe: name it, keep it, do not pretend the others do it.
| Module | Shared | Own recipe (ingredients to make showable) |
|---|---|---|
| enhancement (content) | prepare, retrieve, generate, self-check, settle | template DSL and per-template rules (App V5); research mode as a variant; multi-version copy as three jobs or one job with a versions param; outputs transform to a load-sheet shape |
| image generation | generate, self-check, settle | prompt built from context fields; reference-image search (Oxylabs) as a retrieve recipe; render with OpenAI gpt-image-* or Gemini; validation votes as a judge variant; artifacts to the module's own bucket, not S3 in another region (canon/05 lapse) |
| attribute normalization | retrieve (accepted values snapshot), generate, self-check, judge, settle | per-tenant accepted-values snapshot as reference data; net-new value detection as a needs_review reason; aggregator over many sources as a run_batch recipe |
| part-type matching | retrieve (taxonomy snapshot), generate, judge, settle | tenant taxonomy vs PCdb as a variant; candidates list on needs_review; a judge.parttype variant when self-report is weak (owner: self-report "tends not to be very good in some cases") |
Prompts and templates: the one place the words do not pick#
Prompts are versioned text a model reads. They behave like reference data
(published, versioned, the version recorded on the outcome) and like system
config (they change with the payload's code and its evals). The recipe that
has worked (App V5's PROMPT_MGMT_ENV, its prompt management behind a key)
is: prompts are versioned artifacts the payload loads by name and version,
the version is stamped on every outcome, and a prompt change is a payload
minor bump; per-tenant template text (a customer's copy template) is tenant
config passed in params, never baked into a prompt. Write which one a
given text is in the module's README, because the vocabulary will not.
Evals are part of the module#
An agentic payload without an eval set is a guess with a deploy. The minimum
is a fixed set of items with expected outcomes (or acceptable ranges) run in
env: local by the conformance CLI as an extra group, reporting pass rate,
confidence calibration (self-report versus expected), cost per item, and
duration. A change to a prompt, a model, or a variant default runs it before
merge. Where the eval items come from: resolved review items are the
teaching loop the business note wants, and the app is where they accumulate;
the module ships with a seed set and grows it from resolutions the app hands
back.
What is bespoke, and how it is kept#
A tenant-specific rule inside a payload (JEGS's title format, Zeder's
banned words) is bespoke: it goes in a labelled place (bespoke/<tenant>/
in the module or, better, in the tenant's params passed by the app), it is
named in the instance doc, and it has a review date at which it becomes a
variant, a param, or a rule everyone gets. The App V5 lesson is that the
hardcoded version worked and cost architectural headroom later; the recipe
is to keep it and label it, not to forbid it.
Order of work for one agentic module#
- Manifest first: params, item, output schemas; codes with labels; variants (model, research source, judge); meters; outputs.
- Reference data as a snapshot with a version, loaded through
ctx.reference; prompts as versioned artifacts. - The payload's
runthrough the layers above, every outside call behind aContextverb. - A seed eval set and its conformance group, green in
env: local. - Instance doc with the module's recipes named under "Own", and its Lapses.
Do-nots#
- Do not import a search client or a tracing client in a payload; reach
them through
Context. A model client may be imported, and every call through it is wrapped inctx.traceand, for a vendor with a limiter,ctx.guarded(../contracts/runner-verbs.md), which is what makes the call visible and budgeted. - Do not return a confidence you did not compute or receive.
- Do not guess when unsure;
needs_reviewwith a reason and evidence. - Do not bake a tenant's template into a prompt; pass it in
params. - Do not ship a prompt or model change without running the eval set.
- Do not smooth a module's own recipe into another module's; name it, keep it, make its ingredients showable.