The site is public and has no login because it currently has no input. It renders telemetry that this pipeline produced about itself. The moment any input path is added, that decision gets revisited rather than inherited.
Because it is public, telemetry records carry shape and never content: byte counts, line counts, and a short sha256 prefix, but never file bodies, prompt bodies, or model output. The pipeline needs the content. The site does not get it.
Every bug found while building the first component was a silent-failure bug — the exact class this pipeline exists to eliminate. That is not a coincidence worth hiding.
ripgrep's default multi-threaded search emits matches in completion order. Two identical queries against an unchanged repository returned the same hits in different orders.
Every downstream guarantee rests on replay. If the same input can produce a different ordering, the result hash changes, the packer's output changes, and a run cannot be compared against itself. Determinism is not free and is not the default.
cmd = ['rg', '--json', '--sort', 'path', '--no-require-git']ripgrep exits 2 if any error occurred during the search, including a single unreadable file in an otherwise successful run. The first version of this code raised on every 2.
Treating a soft read error as a hard failure aborts runs that actually succeeded. The correct rule: exit 2 is fatal only when it also produced no hits. Unreadable files are counted into telemetry so a caller can tell 'nothing there' from 'could not look'.
if proc.returncode == 2 and not hits:
raise RuntimeError(...)With --no-messages set, a mistyped search root produced a bare 'ripgrep failed:' with no reason. The underlying error was 'No such file or directory'.
This is exactly the silent-failure class the project exists to remove, reproduced inside the project's own first component. A deterministic component validates its inputs and fails loudly. There is now a precondition check that names the path.
if not root.is_dir():
raise NotADirectoryError(f'search root does not exist: {root}')A search of a non-repo project tree returned 1573 hits across 180 files, all of them vendored dependencies. The project's own source was 59 files.
The fix works, with an honest caveat: for a differently-shaped query the same exclusion changed nothing at all. Whether vendor noise dominates depends entirely on what is being searched for, which is an argument for measuring retrieval rather than assuming it.
Telemetry records carry reserved keys. A caller passing a field named 'kind' would overwrite the record's own kind and corrupt the log.
kind is positional-only, so a colliding keyword cannot be expressed. Any other reserved name gets a trailing underscore rather than being dropped or overwriting: dropping loses data, overwriting corrupts, suffixing keeps both and makes the collision visible.
def record(self, kind: str, /, **fields): ...The site's headline metric — how many queries produced an unstable result hash across runs — reported a failure on its first build. There was no failure. The metric grouped calls by the search pattern alone, so the same pattern run against two different repositories looked like one query returning two different answers.
A monitor that cries wolf gets ignored, which is how a later real failure goes unnoticed. Two calls are comparable only when every input matches: pattern, root, globs, and every flag. The identity function is now explicit rather than assumed.
return (pattern_sha, root, globs, vendor_excluded, fixed_string, ignore_case)The server block set four security headers, and a small location block below it set Cache-Control for HTML. The result was that every page on the site served with no Content-Security-Policy at all. The config was valid, nginx logged nothing, and the site looked completely healthy.
add_header directives are inherited from the enclosing block only if the inner block defines none of its own. Adding one header at a narrower scope silently discards every inherited one. Keep all add_header directives at a single level, and verify headers on a real response rather than reading them out of the config.
curl -sI https://craiger.dev/ | grep -i content-security-policyImmediately after `systemctl reload nginx` returned success, every route on the new site returned 404. Nothing was wrong. Seconds later the same requests returned 200.
nginx reloads gracefully: the command returns as soon as the signal is delivered, while old workers keep serving until their connections drain. Any check that runs in that window tests the previous configuration. It is a small thing, but it is the same shape as every other bug here — a success signal that arrives before the thing it claims to confirm.
ast-grep exits 0 when a pattern matched, 1 when a valid pattern matched nothing — and 0 again when the pattern was malformed, emitting only a warning on stderr. So a typo'd query reports success with zero results, while a correct query that legitimately finds nothing reports failure.
The exit code is inverted relative to the thing you actually care about, which is whether the question was asked properly. The stderr warning about an ERROR node is the only signal that a pattern was broken, so it is now treated as a hard failure. 'No matches' has to mean 'I looked and there were none', never 'I could not parse what you asked for'.
if 'ERROR node' in proc.stderr:
raise ValueError('pattern did not parse; the tool exited 0')ast-grep walks files concurrently and emits matches in completion order. Three identical scans of an unchanged tree returned 6908 matches each time, under three different result hashes. Unlike ripgrep there is no --sort option.
So the ordering is imposed in code, by sorting on file, line and column after parsing. That turned out to be the better pattern regardless: layer 1's determinism is a property of a tool we do not control, and layer 2's is a property of a module we do. Where both are available, prefer the guarantee you own.
hits.sort(key=lambda h: (h.path, h.line_no, h.col))The stability metric flagged two queries as non-deterministic. They were not. One of the repositories being searched is a live harness that generates code into a working directory, and it was running during the scan. The hit counts were identical and the paths had moved — a rename, which leaves modification times untouched and is invisible to any check based on them.
Comparing two result hashes only means something when both runs searched the same code. Every retrieval record now carries a fingerprint of the files that produced its hits — path, size and modification time — so a changed result is attributable to an edit rather than left as a mystery. Records made before the fingerprint existed are counted as unattributable, which is a third answer alongside pass and fail, and an honest one.
identity = (layer, mode, lang, kind, pattern_sha, root, globs, flags, corpus_sha)An index built at one moment describes the code at that moment. Asked about code that has since changed, it does not degrade gracefully — it returns a definition at a line number that has moved, which is worse than no answer because it looks like one.
Every index now carries a fingerprint of the files it was built from, and every query checks it first. The check runs in two stages: a cheap comparison of sizes and modification times, and then, only if that disagrees, a comparison of the actual bytes. The second stage exists because modification time moves when content does not — restoring a file to its original contents, or checking out the same code from git, would otherwise condemn a perfectly valid index to a rebuild.
if stat_fingerprint() == known: return # untouched
if content_fingerprint() != known: raise StaleIndexIndexing this host's mail project succeeded, reported no error, and covered 6 of its 48 TypeScript files. The root tsconfig.json includes only scripts/**/*.ts; it is a monorepo whose real source lives under packages/ and apps/. The tool did exactly what it was configured to do.
A partial index cannot be told apart from a complete one at the query layer. Every answer it gives is correct, and every answer it cannot give looks identical to 'that symbol does not exist'. So the index is now compared against an independent census of source files, taken with a different tool, and a low coverage ratio is an error rather than a result. A partial index is fine when it is chosen. It must never be inherited by accident.
coverage = len(indexed & census) / len(census)
if coverage < min_coverage: raise PartialIndex(...)Line numbers are 1-indexed everywhere in this pipeline, matching editors, tracebacks and every tool involved. Python's list slicing is not. A hit at line -5 sliced from index -6 and silently returned three bytes from the END of the file. A hit at line 0 returned content from the top of a file for a line that does not exist. A hit past the end of the file produced a chunk of zero bytes that still occupied a slot in the bundle and emitted a header with nothing underneath it.
None of these raised. All three produced a bundle that looked structurally fine and contained the wrong code, which is strictly worse than a crash. Line numbers are now validated before they can reach a slice, and every rejected hit is counted by reason rather than quietly dropped — a hit that cannot be turned into a chunk usually means the code moved between searching and packing, which is worth knowing.
if c.line_no < 1: skip('line_below_1'); continue
if c.line_no > len(lines): skip('line_past_eof'); continueThe packer worked correctly on the first real run, across every layer, with sensible output. The three bugs above only appeared when it was fed deliberately hostile input: a deleted file, a line past the end, line zero, a negative line, a malformed key, a path containing a colon.
Working on the happy path is the weakest possible evidence that something is correct, and it is the evidence that feels most convincing. Every bug on this page was in code that had already produced a plausible answer. That is the argument for the whole project in miniature: plausible output is not a success signal.
The spend limit was built on the cost figure the gateway returns with each response. It works on one route and not the others: calls through the Anthropic and automatic routes came back with no cost field at all, so every one of them was recorded as costing nothing. The budget could never trip, on a pipeline whose central component is a repair loop — precisely the shape of program that spends without bound while looking busy.
A call is now never recorded as free. The reported figure is used when it exists and the call is priced locally when it does not, from a table that will go stale — one model on it reprices four days from now — and an unrecognised model is charged at the most expensive rate on the table. Guessing high makes an unknown model exhaust the budget too quickly, which is an annoyance. Guessing low lets it run unbounded, which is not. Token and call counts back it up, because those cannot be mispriced.
if reported > 0: cost = reported\nelse: cost, known = price_call(served_by, tokens_in, tokens_out)The first real planning run was rejected by the contract validator for naming five symbols that did not resolve. All five existed. The plan called one of them Telemetry.record; the symbol index writes the same thing as a package in backticks, a slash, the class, a hash, and the member name with parentheses. The comparison was a literal substring test, so it rejected a correct plan over punctuation and charged a full second planning call to fix it.
The gate did exactly what it was written to do, which is the problem. A check that converts a non-error into a paid retry costs money on every run and teaches whoever reads the rejection to distrust the gate — and a distrusted gate gets bypassed. Symbol lookup is now tolerant of notation while remaining strict about existence: invented symbols are still rejected. Rerunning the same plan afterwards took one attempt instead of two and cost half as much.
'Telemetry.record' -> `core.telemetry`/Telemetry#record().The repair loop ran five iterations, made ten model calls, and fixed nothing. The loop controller reported that the model had stalled. Every single patch the model produced was correct. The workspace gate was applying them with a three-way merge, which needs the blob hashes a real git diff carries in its index line — and a model-generated diff has no index line at all, so the merge failed on patches that applied perfectly well without it.
Three-way application is extra tolerance for diffs that can support it, not a default. It is now used only when the diff actually carries blob metadata. The advice to prefer it came from the critique this whole project started as, which is a reasonable reminder that being right about a principle and right about its application are different things.
return ['apply', '--3way'] if _has_blob_metadata(diff) else ['apply']While the patches above were being refused, the loop controller was reporting STALLED — its verdict for a model emitting the same patch against the same failure over and over. Nothing had stalled. No patch had ever been scored, because none was ever applied. The controller was fingerprinting empty strings, which are identical to each other, so every iteration looked like a repeat of the last.
STALLED is a claim about the model's reasoning. The truth was a claim about this pipeline's plumbing, and acting on the first would have meant escalating to a more expensive model to solve a problem it could not have solved. Iterations that produce nothing to judge are now their own state, excluded from the comparison, and a run in which nothing was ever scored ends with a verdict that says so rather than blaming the model.
if no_output: state = State.NO_OUTPUT # not stalled — never scoredStage 0 assembles a ranked set of excerpts, and the implementation stage was handed the same bundle the planner got — 471 bytes across seven fragments. A unified diff has to quote its surrounding lines byte for byte. Given fragments, the model reconstructed the context it had not been shown; the reconstruction was plausible and wrong, and every patch was rejected for a mismatch that read like carelessness.
The two stages want different things from the same repository. Planning asks what exists and where, which excerpts answer well. Implementation asks what is exactly on line eight, which they cannot answer at all. Files a task will edit are now read whole and quoted verbatim, with the bundle kept as background. After that change the same bug was fixed on the first iteration, in two calls, for two cents.
Writing a telemetry record as `record('x', diff_sha=digest(d), **{f'diff_{k}': v for k, v in shape(d).items()})` raises, because shape() already yields a sha key and the expansion produces diff_sha a second time. This was written, hit, and fixed three separate times in three different files.
Fixing it a third time at the call site would have guaranteed a fourth. A helper now does the prefixing and returns one dict, so the collision cannot be expressed. The general form: when the same error recurs across unrelated code, the bug is in the shape of the API rather than in the person using it.
tel.record('implement_ok', **shaped('diff', diff))