When the Code Outruns the Architecture

A lot of articles I read about AI-assisted software development preached the same message: iterate quickly, let AI accelerate experimentation, embrace agile. I did almost the exact opposite.

I've spent the past few months building EntryIdP, a biometric-first identity provider — one where end-user login happens through face liveness rather than a password or one-time code, and which issues standards-compliant OpenID Connect tokens to relying parties (the applications that outsource their login to it) across multiple tenants. It's the kind of system where a vague requirement or an unchecked assumption doesn't produce a minor bug. It produces a security gap that stays quiet until someone finds it. So instead of the fast, iterative approach every AI development article recommends, I built it the old way: full specification first, phase gates, formal sign-off before anything moved to the next stage. With AI writing a large share of the code.

If your first thought was “that's waterfall”, hold it for a second, because the label does my arguing for me and gets the mechanism wrong. Classical waterfall — the thing Royce described in 1970 and that the industry spent thirty years recovering from — sequences phases and freezes the downstream deliverables. What I ran freezes the governance rules and lets the deliverables move: specify before you build, gate before you advance, sign off before you call anything done, and reconcile the record against reality continuously. Specification-led, gated, continuously assured. Scope moved considerably over the build. What never moved was the way scope was allowed to move. The process stayed rigid; its contents stayed negotiable.

That distinction matters because this framework earns its place for a reason that inverts the usual objection to it. Traditional waterfall failed because humans couldn't predict the future: by the time a year-long plan met reality, the plan was wrong and the process had no legitimate way to admit it. AI-driven gating works because the bottleneck has moved somewhere else entirely. The AI executes far faster than anyone can reason about the consequences — a subsystem can exist, complete and plausible-looking, before you've finished deciding whether it should. The gates aren't there to slow the thinking down. They're there to stop the code outrunning the architecture.

It's worth being precise about which existing practice this is filling in for, because three of them each hit a wall at a different place. Agile and XP assume human coding latency supplies the reflection time — the hours spent typing an implementation are also the hours you spend noticing it's the wrong implementation. AI removes that speed limit, and the pause where architecture review used to happen quietly disappears with it. Classical waterfall assumes requirements are predictable a year out, which they aren't, which is exactly why scope has to stay negotiable. And static analysis and CI validate syntax, types, and unit coverage — all genuinely useful, none of it able to catch the failure mode that actually cost me time: semantic drift between the implementation, the documentation, and the public claim about what the product does. Nothing in a standard pipeline compares a marketing page to a test suite. That third gap is what most of this post is about.

I expected the discipline to feel like a compromise — process tax paid against the speed AI is supposed to give you. It didn't work out that way, and the reasons are worth explaining to anyone weighing up how much process to keep when AI enters the picture.

The specification came first, deliberately

Before any implementation work began, the project had a full set of specification documents: system architecture, API contracts, token claim structures, tenant onboarding rules, key rotation policy, and a set of architecture decision records — short, dated documents each settling one foundational technical choice and recording why. Alongside these sat a formal project plan, explicit about the operating model:

No downstream phase starts until the previous phase passes formal exit criteria.

The plan defined gates — before implementation begins, before beta, before general availability — each with entry and exit checklists requiring sign-off across product, architecture, security, and QA. In practice, this meant every instruction I gave an AI coding assistant operated inside constraints that had already been decided. The agent wasn't inventing the shape of the system as it went. It was filling in a shape that had already been agreed.

This matters more with AI in the loop than without it, for a simple reason: AI is very good at producing a plausible answer to a question nobody has properly asked yet. A human engineer facing an underspecified requirement usually pauses and asks for clarification. An AI assistant, left to its own devices, tends to make a confident, reasonable-sounding assumption and proceed. Specifying first removes the opportunity for that assumption to be made silently.

There's a formal name for what a specification is doing in that arrangement, and it's older than any of this. A written pre-condition, post-condition, and invariant is a contract — a bound on what a component is permitted to be. Point that at a generative model rather than at a compiler and it does the same job: it constrains the solution space before the model starts searching it. “Filling in an agreed shape” isn't a metaphor for good prompting. It's the mechanism.

The lessons AI taught me, the hard way

None of this stopped AI from getting things wrong. What it changed was how long “wrong” stayed wrong before something caught it. Looking back across the build, every failure worth mentioning traces back to the same root cause: a confident statement, from the model or from me, that hadn't actually been checked against reality.

A few examples:

• The same bug came back repeatedly. A page kept breaking after releases and was patched several times over, because each fix addressed the symptom rather than the underlying cause. I never stepped back to ask why the same fault kept returning, and neither did the AI assistant proposing the fixes, which treated each occurrence as a new, unrelated problem rather than a repeating pattern.
• Public-facing copy overstated a security claim. A description of the security model overstated a guarantee — describing something as continuous protection when it was in fact a point-in-time check. For an identity verification product, that's a serious category of mistake to get wrong in public, and it was corrected quickly once I actually checked the claim against the system.
• Developer documentation described something that didn't exist. A setting was documented that had never been implemented, and pointed integrators toward the wrong place for security-relevant configuration. Anyone following it as written would have built something that failed quietly — no error, no warning, just a silent gap.
• The change log couldn't be trusted. Several entries described work that had never happened. Two claimed a login feature had been built when, in reality, only a web page had been edited. Once the record of what changed is unreliable, everything that depends on that record inherits the unreliability.
• Decision records drifted from what had actually shipped. Some described completed work as “not started.” Others described early mockups as finished features. Bringing the record back in line with reality took several dedicated cleanup passes.
• The same rule, written in two places, disagreed with itself. A configuration rule about environment-specific behaviour existed in two separate documents. When they diverged, the same error shipped twice, because there was no single place to check.

Every one of these is the same failure wearing a different outfit: something stated with confidence that nobody had verified. That's not a criticism unique to this project's use of AI — it's close to the central risk of working with AI at all. Models are fluent. Fluency and correctness are not the same thing, and the gap between them is exactly where these failures live.

The uncomfortable half of that is that the gap needs a human to stay open, and I was that human. Human factors research has a name for it: automation bias and complacency — under competing task load, operators monitoring an automated aid systematically under-verify its output, and imperfect aids then produce both the errors of omission you'd expect and errors of commission you wouldn't. That literature is decades old and was written about flight decks, not coding agents, which is rather the point. None of the failures above required an unusually bad model. They required a reviewer with six other things to do and a plausible-looking diff in front of him. So the gates are not primarily a countermeasure to the model's error rate. They're a countermeasure to my review behaviour under load — which is why they have to be mechanical, and why “I'll check it carefully” is not a control.

The gates' contribution wasn't preventing the model from making these mistakes. It was ensuring each one landed somewhere it could be caught, rather than shipping straight into production disguised as progress.

Specifications need policing, not just writing

Writing a good specification on day one is the easy part. Keeping it true weeks later, after hundreds of AI-assisted changes, is where most projects quietly fail — the documentation stays behind in a folder nobody opens, while the system becomes something else entirely.

I built a reconciliation step into the process specifically to prevent that: a recurring check comparing documentation against the actual code and test state, correcting the gap whenever one opened.

It's worth being concrete about the mechanism, because this is the part I'd actually recommend copying. The build ran on Claude Code, driven by sixteen repo-local commands — one per delivery phase, plus separate ones for authoring a decision record, refreshing the roadmap, and reconciling the record. Two of those commands do the reconciling, and the split is deliberate: one audits the code and produces the ledger, the other audits the documentation against that ledger. Both are things you invoke, not calendar reminders you ignore. Three rules are written into the commands themselves rather than left to the model's judgment. Code and passing tests are the only source of truth. No status claim survives without file:line or test evidence behind it. Anything that is a stub, hardcoded, or a TODO is marked as such and can never be described as built.

Roughly, in order:

audit code → ledger
  1  sha ← the "last audited" commit stamped in the ledger header
  2  changed ← git log --oneline sha..HEAD ; git diff --stat sha..HEAD
  3  for each subsystem touched by `changed`:
  4        locate the implementation and its tests
  5        status ← built + test-covered        requires file:line AND a passing test
  6                 built but partial/unguarded
  7                 stub / hardcoded / TODO / absent
  8  grep code/ for new stub | TODO | NotImplemented    ← regressions outside the diff
  9  for each status that moved, reconcile the matching public product tile:
 10        ledger says built + covered   → the tile may say "live"
 11        ledger says anything else     → the tile may not say "live"
 12  stamp the ledger header with today's date and the audited sha

audit ledger → docs
 13  for each canonical doc, any claim the ledger doesn't support is the doc's error → fix the doc
 14  unless the code violates a documented constraint → then the code is the error → raise it,
           never document it away

Line 14 is the one I'd underline. Without it, a reconciliation pass “resolves” every disagreement by rewriting the specification to match whatever the code happens to do, which is drift with extra steps.

What it produces is an as-built ledger: one status marker per subsystem — built and test-covered, built but partial or unguarded, or stub/absent — stamped with the commit it audited, so the next run diffs from that point and only re-audits what actually changed. The step that matters most, though, is the last one: the ledger is then checked against the public product page, which is not prose but a list of capability tiles each carrying a machine-readable status. Fifty tiles, twenty-five of them currently claiming “live”. A tile may only claim that if the ledger has the underlying subsystem marked built, with evidence. That specific check is what caught the overstated security claim above. Not a person reviewing copy — a process refusing to let a marketing tile assert something the test suite didn't support.

None of the underlying ideas are new. Software architecture has known about drift and erosion for thirty years, and there's a mature body of work on conformance checking and automated drift detection — machinery for verifying that an implementation still obeys its intended architecture. What's different here is the direction the check points. Conventional conformance checking validates code against the architecture. This validates the documentation and the customer-facing claims against the code and its passing tests, and treats a marketing page as an artifact that can fail an audit. In a build where the documentation is itself largely AI-generated, that's the direction the drift actually travels.

For scale: about 19,300 lines of application code (excluding generated database migrations) and 13,700 lines of test code across 451 tests, thirty-three decision records, nine architecture decision records and sixteen runbooks, across 506 commits.

The whole operating model, in one picture:

The two loops are the argument. The outer one is how new scope enters legitimately — never by a subsystem simply appearing, always by going back through specification and a gate. The inner one is how the record stays honest: reality flows back into the specification, rather than the specification slowly becoming fiction while the system moves on without it.

None of that is sophisticated. It's a checklist with the authority to say no, pointed at the project's own AI-generated documentation rather than at the code. But it's the piece I'd point to if asked why upfront discipline held up through months of AI-accelerated change instead of decaying into the usual gap between what's documented and what's built. Drift still happened, often enough to need a dedicated process. The difference is that it always had somewhere to go once it was found.

The plan stayed strict on process, flexible on scope

This is where the distinction I opened with did its real work. The common failure mode of waterfall is planning an entire system upfront and then refusing to adapt when reality disagrees — the plan treated as a promise about the future rather than a framework for governing decisions. That isn't what happened here. The gates were quality control, not commitments about what the system would eventually contain.

The foundational specification documents were written early and stayed largely fixed — stable references, rarely revised. But new scope was added over the course of the build — an administration console and self-service tenant onboarding, for instance — without breaking the existing structure. Each addition went through the same specify-first, phase-gated approach as everything that came before it. The plan held its shape. It didn't hold its contents fixed forever.

The decision-record system evolved the same way. It wasn't designed as two separate tracks from the outset. A narrow, technical record of architecture decisions existed from the start. A broader log — covering compliance questions, vendor choices, tenant-level configuration decisions — emerged only once it became clear that a different kind of decision needed a different kind of record. The formal template and process for it arrived after the pattern had already proven itself useful, not before.

What actually held together, in other words, wasn't a complete specification of the whole system decided in advance. It was a consistent way of deciding things — nothing counts as agreed until it's written down, nothing moves to the next phase without sign-off — applied firmly enough that new scope always had somewhere legitimate to go.

Where the discipline cost something

None of this came free, and it's worth being honest about where it slowed things down for no good reason, and where it slowed things down for a very good one.

A regulatory review held back a third-party integration for longer than the engineering work itself took. I knew what the integration needed to do; the gate wasn't protecting against uncertainty in the code, it was protecting against a compliance answer that hadn't arrived yet. That's a legitimate cost, but it didn't feel that way in the moment.

Elsewhere, a self-imposed rule that infrastructure changes had to be defined as code, tested, and repeatable — no shortcuts — slowed early progress considerably. I've been on the other side of this before, frustrated at how cautiously infrastructure teams tend to move. Building it myself was a useful correction: that caution isn't process for its own sake, it's what repeatability actually costs. Once that was clear, I restructured the work to front-load infrastructure changes rather than paying that cost repeatedly throughout the build.

The most instructive cost, though, came from a moment I skipped the process rather than one where the process slowed me down. At one point I had AI reconstruct a specification from an existing feature built earlier, in a previous system, and reviewed the result too quickly — a glance rather than a proper check. The gap between what I'd approved and what the feature actually needed to do surfaced later and cost real rework to fix. That failure doesn't belong to “AI got it wrong” or “the process is too slow.” It belongs to the exact failure the process exists to prevent, happening anyway, because that particular check was nominal rather than real. It is also, precisely, the automation complacency above — catching the author of an article about it, in the act of writing the system it describes.

And some problems simply sit outside what any amount of specification can reach. Liveness detection — checking whether what's presented to a camera is a live subject rather than a photograph, video, or mask — resists upfront specification almost entirely. Those issues get found through hands-on testing and direct observation, not through better documentation. It's worth saying plainly: a good process disciplines the parts of a system that can be specified. Some parts can't be, and you handle those the way engineers always have.

It's also why, for liveness specifically, I chose to build on a detection capability that had already been through independent, standardised testing rather than relying on my own confidence in something I'd built myself — my cloud provider's managed face liveness service, evaluated against Level 1 and Level 2 iBeta Presentation Attack Detection conformance testing to ISO/IEC 30107-3. Some things you specify. Some things you verify against an outside standard, because your own certainty isn't evidence.

What this evidence does and doesn't support

Applying this article's own standard to this article: it is one build, and there are two things it cannot tell you.

It's one developer and one agent. Those entry and exit checklists requiring sign-off “across product, architecture, security, and QA” were four hats I wore, not four people I convinced. That makes the discipline easier in one respect — no negotiation — and much weaker in another, because a gate you administer to yourself is only as good as your willingness to fail your own work, which is what the “nominal rather than real” story above is about. Scale it to several engineers running several agents concurrently and you get a class of problem this setup never faced: divergent implementations of the same intent, landing in parallel, with drift appearing at the merge rather than in any single change. A single as-built ledger has no obvious answer to that, and I'd be interested in one.

There's no control arm, so there's no defect-density figure. The honest comparison — defect density under gated delivery versus the same system built by unguarded AI generation — needs a baseline this project never ran, and I'm not going to quote a number I'd have to invent to produce. That would be the exact failure this whole post is about. What I can report is what's derivable from the repository: 451 tests against roughly 19,300 lines of application code, a test-to-code ratio of about 0.7 and roughly 23 tests per thousand lines; and, of the six classes of failure listed earlier, four were caught by the reconciliation step — the overstated security claim, the phantom setting, the untrustworthy changelog, the drifted decision records. The other two weren't. The recurring bug was caught by its own repetition finally becoming obvious to me, and the duplicated configuration rule was caught by shipping the same defect twice. That ratio is the actual finding, and note what it isn't: the process didn't prevent four of six, it detected them after the fact. The claim available here is about detection, not prevention. Directional, not conclusive. It's a case study, not a trial.

What shipped, and what I'd take into the next build

EntryIdP is now officially launched to its production environment: OIDC and OAuth 2.0 Authorization Code flow with PKCE, tokens signed with a hardware-backed key management service using RS256, and claims resolved for the requesting client's tenant.

The specific value of gating in an identity system is that it converts security intentions into things that can't quietly stop being true. Four examples, all of them written down before they were built and all still enforced: machine-to-machine credentials are rejected at the protocol-handler level and that grant type is never registered, so “we don't do M2M” is a property of the code rather than a habit. Signing keys never exist as local key material — the ephemeral-key convenience that every framework offers for local development is prohibited in every environment, including my laptop. Authorization codes are single-use with a sixty-second lifetime, deleted on use. Redirect URIs are exact-match only: no wildcards, no prefix matching.

The one I'd single out is PKCE. The proof-key check — the step that stops an intercepted authorization code being redeemed by someone who didn't start the flow — initially existed as a stub, as such things do early in a build. What kept it visible was the ledger rule that a stub can never be described as built: for as long as the verifier check wasn't real, no document, roadmap, or product tile was permitted to say the flow was secure. It got finished because it couldn't be forgotten, not because I remembered it.

The habit that caused the most damage across this build wasn't a bad architectural decision or a slow approval gate. It was confidence without verification — a model, or an engineer, stating something as though it had been checked when it hadn't. The process didn't stop that habit from occurring. It made sure every confident, unverified statement eventually had to sit next to the thing it claimed about, and be corrected when the two didn't match.

That's a narrower lesson than “gates beat agility for AI-assisted development,” and I think it's the more useful one. Take the governance, not the straitjacket: decide upfront how decisions get made and recorded, hold that discipline consistently, and let the plan absorb whatever the project actually turns out to need rather than pretending you predicted all of it on day one. The gates aren't a prediction about the system. They're the only thing keeping the code from arriving somewhere your architecture hasn't been yet.