Engineering lab
After the happy path
Three independent systems about what happens when things go wrong: repeated deliveries, a model regression, and a report that counts the same person twice.
Independent portfolio projects · Synthetic data
Follow the installed operations handoff One event, three tools, exact recorded evidence.
RelayBox
A signed webhook inbox with a durable SQLite job lifecycle. Delivery IDs deduplicate matching payloads, conflicting reuses are rejected, and expiring leases let a new worker recover interrupted work.
A fencing token prevents an old worker from acknowledging a job after another worker has reclaimed it.
Queue visibility 0.4.0 separates ready/delayed work and active/expired leases without mutating the inbox. Recovery 0.3.0 adds verified snapshots and new-path restore; retain receipts separately. Worker 0.2.0 introduced bounded processing and canonical receipts. Receipt publication and inbox acknowledgment remain separate transactions.
- queuedPersist the accepted delivery
- leasedClaim a bounded worker lease
- queued (retry)Recover with another attempt
- succeeded / dead_letterRecord the outcome explicitly
At-least-once work requires idempotent handlers. A durable inbox cannot make an external side effect happen exactly once.
Review the code Reproduce a stale worker
Run the boundary
From a RelayBox checkout with Python 3.11+, run this focused test. It uses a temporary database and a controlled clock.
python3 -m unittest tests.test_store.StoreTests.test_stale_worker_is_fenced_after_reclaim -vExpected: one passing test. Worker B reclaims an expired lease; worker A's old token cannot acknowledge it.
Follow the decision
Read claim and _complete together. The SQLite write transaction selects one owner, while the current token and expiry guard completion.
A process can still finish an external side effect after losing its lease. The consumer must make that effect idempotent; this database cannot roll it back.
TraceBench
An offline evaluation toolkit for recorded model outputs. Explicit exact, contains and JSON-subset checks turn English and Mandarin fixtures into reproducible results. Baseline comparisons identify which cases regressed.
Supplied latency observations and explicit token pricing remain separate from correctness. No provider keys or model calls are needed to reproduce the examples.
Reusable CI 0.3.0 retains JSON, Markdown and HTML evidence before applying the regression gate. The current live report keeps the searchable, printable results introduced in HTML 0.2.0. Its two regressions are deliberate; every output, timing and price is synthetic.
A passing contract is evidence about those cases. It does not establish general model quality or safety.
Review the code Find a pass-to-fail regression
Run the boundary
From a TraceBench checkout with Python 3.11+, run the baseline comparison test. It joins cases by ID, independently of output order.
python3 -m unittest tests.test_evaluate.EvaluationTests.test_baseline_detects_regressions_and_improvements_by_id -vExpected: one passing test. A new failure remains a regression even when another case improves.
Follow the decision
Trace suite fingerprinting and the comparison of per-case pass states. A changed contract requires a fresh baseline, so unlike suites cannot quietly produce a reassuring comparison.
These checks measure explicit fixture contracts. Recorded timing and token usage are supplied observations; a passing result does not establish broad model quality.
Impact Ledger
A nonprofit reporting workbench with a typed frontend, a Python API and a SQLite store. Filter synthetic program records, distinguish unique participants from attendance, and compare session spending with budget.
Sessions and attendance have separate grains, so a busy event does not multiply its own expenses. Imports validate before committing, and accessible tables keep the numbers available alongside the charts.
Import preview 1.2.0 adds optional advisory CSV validation in the local API; preview reserves no IDs and import revalidates atomically. The hosted bundled synthetic snapshot retains the equal-length comparison and print report from Reporting 1.1.0. Set both dates, apply, then compare. Persistent imports need the local API; activity counts do not establish causal impact.
- People reached
- Distinct participant IDs in the filtered attendance.
- Attendance
- One participant's visit to one session.
- Program spending
- Count each session's actual cost once.
- Filter scope
- State the selected program and date range.
Review the code Check the reporting grain
Run the boundary
From an Impact Ledger checkout with Python 3.11+, run the storage tests. The first reporting case checks attendance, distinct people and spending together.
python3 -m unittest discover -s tests -p test_store.py -vExpected: the suite passes. Adding attendance does not multiply a session's spending, and repeat visitors are counted distinctly.
Follow the decision
Inspect the session and attendance tables, then follow _rows into summarize. Costs belong to a session; visits belong to a participant and session. Filters must describe the same reporting population.
Participation and spending show activity. They do not establish outcomes, eligibility coverage or causal impact. Every example record is synthetic.
One installed handoff
From one webhook to one report
This AI-assisted educational drill connects the three released tools through real local HTTP services. Fictional English/Mandarin names, including Alex Chen / 陈安, make the synthetic fixtures readable; the ledger imports participant IDs.
- Accept a signed event. RelayBox persists the delivery, deduplicates an identical replay and rejects a changed one.
- Keep the receipt. A bounded worker writes canonical receipts; an explicit adapter extracts their sessions and attendance CSV bytes.
- Preview, then import. Impact Ledger's advisory preview changes no rows and reserves no IDs. A separate explicit import revalidates and commits atomically.
- Compare exact totals. The selected program and inclusive dates fix the population. Costs count each session once; people count distinct IDs.
- Check the recorded contract. TraceBench checks nine recorded JSON contracts, with no provider calls.
The receipt effect, CSV import and inbox acknowledgment are separate transactions. They do not guarantee exactly-once processing across systems.
| Measure | Current: October 1–31, 2026 | Prior: August 31–September 30, 2026 |
|---|---|---|
| Distinct people | 3 | 2 |
| Attendance | 4 | 3 |
| Participant-hours | 7 | 3.25 |
| Session-hours | 3.5 | 2 |
| Actual spend (USD) | 130.25 | 131.00 |
Participant-hours sum each person's attended session duration; session-hours count each session once. These values match an independently hand-derived fixture oracle.
Make one cent fail
Clean: 9/9, exit 0. A deliberate synthetic mutation of recorded actual cents, 13025 → 13024, gives 8/9, exit 3; only import-totals regresses. Invalid input gives exit 2. This verifies the recorded contract, without a model-quality claim.
Reproduce it with the prepared wheels
From the portfolio repository root with Python 3.11+, follow the README's separate explicit download step first. Set WHEEL_DIR and OUTPUT_DIR to your local paths outside the checkout, with existing trusted parents; the output directory must not exist. This command installs the locked wheels offline in a fresh environment.
python3 -B scripts/verify_handoff.py --wheel-dir "$WHEEL_DIR" --output "$OUTPUT_DIR"How to review the work
Start with the failure
Reproduce the example
Each README starts with a local quickstart and synthetic fixtures. No paid service is needed.
Inspect a boundary test
Trace a stale lease, a missing evaluation output, or a malformed import through the actual test and implementation.
Read the tradeoffs
Architecture notes and verification records explain the supported behavior and the limits of the evidence.
These independent, AI-assisted portfolio builds connect my interests in applied AI, useful interfaces and business operations. English READMEs and Chinese guides support a clear technical handoff. They are educational systems with documented deployment limits.
Back to all projects