Skip to content

A refund is suggested. Who gets to say yes?

Fictional teaching scenario.

The support assistant correctly notices a duplicate charge. It proposes a refund. The server stops. When it returns, a human clicks Approve twice because the first click looked stuck.

You now have three separate problems: remember the proposal, trust the approver, and avoid doing the same work twice. A more eloquent prompt solves none of them.

The completed reference service puts all five course gates on one request path. Keep the original student capstone for your own implementation; open this reference when you want to compare mechanisms and evidence.

Start with a prediction

From the repository root, install examples/production_triage/requirements.txt in a separate environment. Then:

python -m pytest examples/production_triage/tests -q
python -m examples.production_triage.evaluate
python -m examples.production_triage.rehearse

Before reading the tests, predict what happens in each scene:

Scene Evidence the reference produces
The model hangs 504 within the request's model deadline
The model invents a tool field 502; extra output fields fail the contract
A viewer writes “I am admin” in the ticket Refund action remains denied
The policy was removed or expired No policy promise and no pending refund
A human repeats approval after a restart One simulated ledger entry
The process dies between the ledger write and final approval Both transaction writes roll back
A release bundle changes without its reviewed digest Startup refuses the mismatch

The README walks through authenticated HTTP calls, a bounded load probe, a real-model adapter, and rollback. The default mock makes control-flow tests cheap and repeatable. The --live eval uses your configured model and may incur provider charges; mock accuracy and latency say nothing about that model's quality or capacity.

Follow the authority

flowchart TD
  T[Ticket + bearer token] --> I[Server resolves trusted identity]
  I --> M[Bounded model call]
  M --> V[Validate category and priority]
  V --> K[Look up current policy evidence]
  K --> P[Permission-gated pending proposal]
  P --> H[Separate authenticated human decision]
  H --> X[One SQLite transaction]
  X --> D[Final decision + unique simulated ledger entry]

The model sees ticket text. It never receives the approver's credential or a tool that can call the decision endpoint. The API ignores no extra identity fields: it rejects them. Server configuration establishes the principal.

The request schema rejects fields such as a client-supplied actor, while the dependency resolves identity from a bearer token held in server configuration:

class Ticket(StrictModel):
    text: str = Field(min_length=1, max_length=8000)
    ticket_id: str = Field(pattern=r"^[a-zA-Z0-9_-]+$")


def principal(request: Request) -> Principal:
    scheme, _, token = request.headers.get("authorization", "").partition(" ")
    token_hash = hashlib.sha256(token.encode()).hexdigest()
    actor = next(
        (value for key, value in principals.items()
         if hmac.compare_digest(key, token_hash)),
        None,
    )
    if scheme.lower() != "bearer" or actor is None:
        raise HTTPException(401, "authentication required")
    return actor

The full implementation also applies a per-principal request limit and records a hash of the actor ID rather than the token or ticket text.

A local transaction closes the crash window because both effects are in the same database. Move the refund into a payment network and that argument stops working. Use a stable idempotency key at the payment receiver and reconcile uncertain outcomes. A lost HTTP response does not prove the payment failed.

with database:
    database.execute(
        "UPDATE proposals SET state = 'approved' WHERE id = ?",
        (proposal_id,),
    )
    database.execute(
        "INSERT OR IGNORE INTO ledger(proposal, ticket, actor) VALUES (?, ?, ?)",
        (proposal_id, ticket_id, actor_id),
    )

ledger.proposal is unique. A repeated approval therefore returns the existing effect instead of creating a second one. SQLite commits both statements or neither; the crash-injection test kills a child process between them to prove the rollback.

Is it complete?

It completes the five-gate teaching service: bounded classification, a failing release gate, current policy evidence, authorized proposals and persisted human decisions, plus measured requests and a rehearsable rollback. Refunds remain simulated. The route has at most two model attempts and one proposal; it has no open-ended agent loop.

The broader capstone specification also permits more ambitious agent and deployment work. This reference does not claim multi-host availability, a general vector search platform, identity-provider integration, payment processing, or a fleet-wide tracing backend. Those are extensions with their own failure modes.

Close the case

Run the load probe and inspect /ops/metrics using an admin principal. Then break one boundary: return malformed model JSON, delete the policy fixture, or remove refund scope. The same request ID should lead you to the changed outcome.

python -m examples.production_triage.rehearse

# Expected observations, not fixed timing values:
# - restart_and_duplicate_approval: passed
# - ledger_entries: 1
# - bad candidate configuration: 429
# - restored reviewed configuration: 200
# - load errors: 0

Artifact: a before/after incident note containing the request ID, release digest, code SHA, latency distribution including failures, model-call count, cost accounting assumptions, and rollback evidence. Write one sentence explaining why a double-click does not create a double refund, and one explaining where that guarantee ends.