Agent docs

Deploying a module

How a module gets from a repo to something a caller can reach, on GCP first, with a Render variant…

How a module gets from a repo to something a caller can reach, on GCP first, with a Render variant at the end. This is the method; canon/12-deployment-and- environments.md carries the rules it implements, and versable-builder/docs/app-patterns/03-deploying-on-gcp.md carries the GCP mechanics (branch-to-environment mapping, /build-info, and four traps that each cost a real build). Read that one rather than expecting it restated here.

Audience: an agent or person standing up a module, or working out why a deploy did not take.

The organising idea is that a module must be provisionable by a script and verifiable by a stranger. Nothing here should require someone to remember a console click, and confirming what is running should not require a credential.

Zero. The environment that needs no deploy#

Before any of this: the module runs in-process with no cloud at all (canon/13-local-dev-and-debugging.md). versable-runner does this with RUNNER_DISPATCH=local, which swaps the Cloud Tasks dispatcher for an asyncio semaphore of the same shape (src/services-api/app/dispatch.py:131-132,174), and it imports with zero environment because every config var carries a default (src/services-api/lib/config/__init__.py:103-122).

That mode is an environment, and it counts toward the three. A module that cannot run locally has two environments and a hope.

One. Provisioning is a script in the repo#

src/services-api/deploy.sh is the reference shape. Ninety-seven lines, idempotent, parameterized entirely by environment variables at the top (deploy.sh:7-16), and it does the following in order:

  1. Enable the APIs the module needs, unconditionally. Re-running is free.
  2. Create the service account if absent (describe … || create).
  3. Create the bucket if absent, with uniform bucket-level access, then bind the service account to it.
  4. Create the queue if absent, then always update its ceilings (deploy.sh:36-41). Note the comment on line 36: max-concurrent = semaphore. The queue's --max-concurrent-dispatches is the module's global concurrency limit and nothing else enforces one, so it is provisioning, not tuning.
  5. Deploy, passing identity, config and secrets in one command (deploy.sh:68-72).
  6. Resolve the service URL and set it back as an env var (deploy.sh:86), because the OIDC audience for the module's own task callbacks is its own URL, which does not exist until the first deploy. The script prints a manual fallback if resolution fails (deploy.sh:80-81).

Step 6 is the one people miss. A module that pushes tasks to itself has a chicken-and-egg dependency on its own URL, and a two-pass deploy is the honest resolution.

Secrets go in by reference, never by value (deploy.sh:72, eight --set-secrets entries of the form NAME=SECRET:latest). Config goes in as plain env (deploy.sh:71). The split is visible in the deploy command itself, which means a reviewer can see at a glance that no secret is inlined.

Terraform is fine where an estate wants it. The rule is that provisioning is code in the module's repo, not a memory of clicks.

Two. Three environments, each with its own identity#

local, dev, prod, plus staging when a module needs a soak.

Per environment: its own service account, bucket, queue, secrets namespace, and issuer client. Not a shared one with a prefix.

The lapse to avoid is live in this estate. runner-service-dev runs as runner-service@, the production service account, so dev can read prod's bucket and neither can be audited apart. canon/12 bans it. The extractor is the worse case: one environment total, which the owner named as a pain point.

Parameterize by environment variable so one script serves all of them:

SERVICE=runner-service-dev QUEUE=pipeline-runner-dev \
BUCKET=versable-runner-jobs-dev PROMPT_MGMT_ENV=dev ./deploy.sh

That is versable-runner's own documented dev invocation (src/services-api/docs/runner-service.md:165-166). The shape is right; the missing piece is SERVICE_ACCOUNT in that list.

Three. Stamp the commit at build time, expose it unauthenticated#

The image build takes the commit as an argument and the runtime reports it:

ARG GIT_SHA → ENV APP_BUILD_SHA → GET /build-info

walmart-mvp passes --build-arg GIT_SHA="${COMMIT_SHA}" (walmart-mvp/cloudbuild.deploy.yaml:77) and serves it back. speedway reports commit, branch, kit and build time. versable-runner has no build identity at all, which is why nobody can currently tell which commit answered a request.

Then make the deploy assert it. walmart's build polls the deployed URL and compares the served SHA against the one it just built (cloudbuild.deploy.yaml:96-112); a mismatch fails the build. Without that step a build goes green on a deploy that silently kept the old revision, which has happened in this estate.

/build-info carries {commit, built_at, branch?, name, version, contract_version}, snake_case, and nothing else; the environment name lives in the manifest's env, not here. It is one shape for a module and for an app, so the same deploy check works on both. Nothing in it is sensitive, so it needs no auth, and confirming a deploy becomes a curl rather than a console login.

Four. Tag images by commit, never by a shared name#

Two environments sharing one image tag is a real defect with a real history here: both walmart targets originally pushed walmart-app:latest, so a preview build overwrote the artifact a production deploy would later reuse with --skip-build. The fix is a per-commit tag (walmart-mvp/deploy.sh:20-22,29), which makes the artifact immutable and lets a rollback name a real thing.

The extraction services show the end state: one image data-extraction/app:47651fadf0c5…, two services running it.

Five. Choose a topology, and say which#

Two shapes are in use, and both are defensible.

One service that pushes to itself. versable-runner enqueues Cloud Tasks that target its own URL. Simple, one deployment, and API capacity and worker capacity are the same number (maxScale 5 covers both). Fine at small scale.

One image, two services, split by role. walmart-mvp runs walmart-api and walmart-worker from one image; the extraction pair does the same with genuinely different shapes per role: api at maxScale 100, 1Gi, 60s timeout, worker at maxScale 50, 2Gi, 300s. The split is what lets an expensive payload stop starving the submit path.

State the choice in the manifest's runtime block, and write down the trigger for changing it: when an expensive payload starves submits, or when worker memory needs diverge from API memory needs. walmart's arq concurrency went from 5 to 2 after image_normalization OOM-killed a 512Mi worker (walmart-mvp/backend/app/worker.py:116-121), which is that trigger firing.

Two Cloud Run specifics worth knowing before you copy them. minScale 1 costs money continuously and buys back the first request's cold start; the extraction pair is the only service in the estate paying it, the api at 1 and the worker at 10 always-warm instances (../instances/extraction.md), which is the number to argue any new minScale against. A pgbouncer sidecar and a VPC connector are what a Cloud SQL or Memorystore dependency costs you; a module on GCS and Cloud Tasks alone needs neither, and versable-runner has neither.

Six. Branches map to environments, once there is CI#

Adopt walmart's mapping (versable-builder/docs/app-patterns/03-deploying-on- [gcp.md](/docs/contract-patterns/gcp):18-33): main previews to dev on every push, release ships to prod, and a deploy-prod-* tag pins a commit or carries a hotfix. A tag rather than a branch push, because the standing rule is one deploy per batch and a push-triggered deploy fires on every commit.

walmart's build also refuses a tag whose commit is not an ancestor of main or release (cloudbuild.deploy.yaml:57-59). That check is worth copying; it is the difference between a tag that pins a reviewed commit and a tag that ships whatever someone had checked out.

Until a module has CI, deploy.sh from a workstation is acceptable, and its output must name the commit it deployed. versable-runner deploys this way today and does not print the commit, which is the gap.

The Render variant#

App V5 is the estate's only non-GCP instance, and the differences are smaller than they look.

Render deploys Docker on push to a tracked branch with a path filter, so branch-to-environment mapping is the platform's native model rather than something you build with triggers: development serves dev, release serves prod, one Render service per role per environment.

Three things change:

  • Concurrency is a deployment number, not code or queue config. App V5 runs 25 worker instances in prod against 5 in dev, set on the Render service. That is easier to change and harder to discover than a ceiling in the repo. If you deploy this way, mirror the number into config so /health/deep can report it.
  • Managed Postgres and Redis replace Cloud SQL and Memorystore, and there is no VPC connector or pgbouncer sidecar to arrange. The connection pooling problem does not disappear, it moves into the platform.
  • The queue must be namespaced per environment or previews steal work. App V5 tags every task with a queue field matched against Config.WORKER_QUEUE, set from the Render-injected branch name, so a preview and its base share one Mongo without draining each other. On GCP the equivalent is a queue per preview, created and deleted by the deploy.

Everything else in this doc holds unchanged: provision by script, three environments with separate identity, stamp the commit, expose /build-info, tag artifacts immutably, and state the topology.

Checklist#

  • the module runs in-process with zero environment set
  • provisioning is one idempotent script in the repo
  • the script creates the queue and sets its named ceilings
  • three environments minimum, each with its own service account, bucket, queue and secrets namespace
  • secrets injected by reference, config as plain env, visibly separate
  • the commit is a build arg and /build-info serves it without auth
  • the deploy asserts the served commit matches the built one
  • images are tagged by commit, never by a shared name
  • the topology choice and its change-trigger are written in the manifest
  • a workstation deploy prints the commit it deployed

Do-nots#

  • Do not share a service account, bucket, queue, or secrets namespace between environments. (runner-service-dev runs as runner-service@)
  • Do not let two environments push the same image tag. Tag by commit. (walmart's preview overwrote the artifact prod would reuse)
  • Do not deploy without a /build-info that names the running commit.
  • Do not call a deploy done because the build went green. Assert the served commit equals the built one.
  • Do not provision by hand. If it is not an idempotent script in the repo, it will be a memory of clicks by the second environment.
  • Do not deploy a module that cannot run in-process with zero environment set.
  • Do not leave a preview sharing its base environment's queue.
  • Do not set minScale above zero without saying what the cold start was costing. It bills continuously.
  • Do not ship a workstation deploy that does not print the commit it deployed.
@versable-git/ui · reference, canon, and method, read in place