Your deploy checklist is a hypothesis, not a fact

A good checklist, applied to the wrong shape

The goal was ordinary convergence work: one env loader, one typed variable registry, one local dev stack recipe, one production-deploy checklist — the same setup already running across a handful of other repos. The checklists were genuinely good. They were full of hard-won specifics: connection-string pooling quirks, build-time versus runtime env timing, which secrets have to be split. Following them felt like following a recipe, which is exactly the feeling worth being suspicious of.

This repo was not shaped like the others. Different frontend framework. Different way of managing database schema. And, most importantly, a completely different mechanism for resolving user permissions than the checklist assumed. The checklist kept telling me to register the access-token hook that grants a role via the JWT. This app has no such hook. It never did.

Applying a good checklist to the wrong shape does not fail loudly. It quietly builds the wrong thing — or, if you are paying attention, it makes you stop and ask “does this actually apply here?” three separate times in one session, for three separate steps.

What the checklist assumed vs. what the app did

The mismatch was not cosmetic. It was the entire authorization model:

• What the checklist assumed: a Postgres access-token hook fires at login, the JWT carries org and role claims, and the app trusts the claim on every request.
• What this app actually does: an OIDC callback resolves the role from a plain env-var mapping, issues an opaque session token server-side, and re-checks that stored token on every request — never a JWT claim.

Neither is wrong — but only one of them is here

Both models are defensible. The point is that building the first one into an app that already, deliberately, does the second would have been a regression dressed up as compliance with best practice. A parallel permissions system nobody asked for is worse than no checklist at all, because it arrives with the credibility of a standard.

So the fix was never “follow the checklist harder.” It was: before implementing any step, trace the actual code path. Grep for the concept the step assumes. Read the real schema. Read the real auth handler. If the assumption does not hold, say so out loud — in the commit message, in a short decision record — rather than silently building the parallel system, or silently skipping the step and hoping it did not matter.

The config write that never deployed

Once the local stack was reconciled against reality instead of the template, the deploy itself surfaced one more assumption I had not questioned: that writing a config value to a managed platform and pushing code would “just” trigger a fresh deploy. On this platform, for this service, it did not.

The env var write sat there, unapplied, until I asked the platform’s own API whether anything had actually changed. It hadn’t. That is a silent no-op — the worst failure mode there is, because everything on your screen says success. Verifying instead of assuming turned it into a five-line fix: trigger the deploy explicitly, then poll it to a real terminal state before calling it done.

Then it broke twice in an hour

The deploy went out clean. Two minutes later a teammate logged in and landed on a blank page. A few minutes after that, login worked but a dashboard threw two 403s.

Both were real, and neither was catchable locally, because both only manifest against the live OAuth provider, from a real browser, on the real domain. This is the category of bug where reasoning from the architecture in your head is the slowest possible approach — you can burn an hour constructing theories about the auth pipeline before you ever look at what the browser actually sent.

So I stopped guessing and asked for a HAR file: the browser’s own record of every request and response for a page load. Two investigations, two exact lines, no application code touched.

What the HAR file said

Bug one: the authorize request came back 400 invalid redirect_uri. Not a typo, not a missing env var. The OAuth provider validates the callback URL against a specific registered list for that client — a stricter, separate check from the general allowed-origins list I had already verified. The domain was on the general list. It simply was not on the client’s own list. One API call to widen it, one API call to prove the authorize request now redirects instead of erroring, done.

Bug two: login worked, but two endpoints returned 403. The HAR showed a role claim of “reader” on the session, and the endpoints correctly denying a reader access to admin-only routes. Not a bug. Working as designed. The real question was a product one — should this specific person be an admin? — and the HAR made that distinction unambiguous in about ten seconds instead of an hour spent wondering whether auth itself was broken.

That second one is the underrated half of the technique. A HAR file that shows the resolved role sitting right next to the denial converts “is something broken?” into “is this the access policy we want?” Those are very different conversations, and only one of them belongs to engineering.

What to ask before you trust a checklist on a new repo

• Which repos was this checklist written against, and is the one in front of me shaped the same way underneath — same auth model, same schema management, same framework?
• For every step that touches auth, schema, or framework internals: have I grepped for the concept it assumes and confirmed the code path actually exists here?
• When an assumption does not hold, am I recording the deviation somewhere durable — commit message, decision record — or silently skipping a step and hoping?
• After writing a config value to a managed platform, have I checked the platform’s own record that a deploy actually ran and reached a terminal state, rather than trusting that the write implied one?
• For any bug that only reproduces against a live browser flow — OAuth, redirects, cookies — have I gotten the HAR file before forming a theory?
• When a 403 shows up, does the HAR show the resolved role next to the denial, so I can tell a broken pipeline from a working policy I disagree with?