From stoic processes to stochastic collaborators
Whenever we acquire a product or service, we enter into a contract. Sometimes it is written, sometimes implied, but it always defines what is being exchanged and what expectations come with that exchange. This is true whether the trade is money for food, attention for entertainment, or trust for competence.
Historically, this could be handled at a human level. But as choice expanded and systems grew more complex, the distance between parties increased. With distance came uncertainty. And with uncertainty came dispute, and sometimes litigation.
In response, we built safeguards. Mechanisms designed not to eliminate human unpredictability, but to keep it within boundaries, to place it on something like train tracks, where outputs could be observed, measured, and, most importantly, relied upon. These systems did not promise perfection but that we would do what we said we would do, and that when we fell short, we would be able to understand why.
Quality systems emerged from this need. They provide accredited assurance that obligations will be approached with care and discipline, and that when failures occur – as they inevitably do – (dashes, mine) those failures can be examined, explained, and learnt from.
The preamble on the ISO 9001 website reads
Implementing ISO 9001 means your organisation has put in place effective processes and trained staff to deliver flawless products or services time after time.
The ISO FAQ defines quality as
the degree to which a set of inherent characteristics of an object fulfils requirements
Both statements rest on an assumption so familiar it rarely needs to be said: that the systems carrying out these processes behave predictably.
Predictable Machines, Imperfect Humans
Before the probabilistic age, we were the fuzzy ones. Quality systems existed to allow humans to be imperfect inside processes that were themselves predictable, supported and monitored by machines that behaved consistently. Governance was human, workloads were human, but the software and hardware beneath them, when given the same configuration and the same inputs, would produce the same outputs.
Humans do not work like that. We phrase things differently each time we utter them, even when we want the same thing. We infer meaning from context, from what is visible and what is not, from tone, from omission. We adapt constantly, often without realising it. We bring cross domain knowledge to every party. And yet, at the point where our actions intersect with the transactional core of a system, we press a button that is always labelled the same, and always performs the same action.
This was the contract between the human and machine. We were allowed to be variable, because the systems around us were not.
Machines were not perfect. Hardware failed and software contained bugs. But their failures were bounded. Their behaviour, once understood, remained consistent. Given the same state and the same input, they would follow the same path. Their imperfection was consistent, and in that consistency, systems could be built, observed, and trusted.
“In God we trust. All others must bring data.”
W. Edwards Deming
The Machine Stops Being Predictable or On Faster Horses
In 2025 the total output of just about anything grew, GitHub reported1 increases in all but one reporting metric (important, see below), with code contributions growing by 25% and Adobe reported2 a huge surge in all document activity, especially where linked to AI. We have supercharged ourselves, a significant proportion of us are using AI to create, draft, summarise, analyse, automate, whatever it was we were creating, drafting, summarising or analysing in 2024. The faster horse is going faster.
A note on Github:
A very telling statistic is that there is a 27% reduction in the number of comments on code being committed to GitHub – we could posit that a significant proportion of the increase code being committed, is from the likes of Lovable, and that there is no actual engineer sat behind it, and certainly no SDLC or peer reviewing *or* we could posit, as I have seen in numerous business, that the engineers embracing AI are producing so much code, the peer review process cannot keep up… For me this is a leading indicator not just in the world of code, but for all blossoming outputs that we are seeing today – there is simply too much to review. The dynamic echoes an older observation by the economist Robert Solow in 1987:
“You can see the computer age everywhere but in the productivity statistics”
Robert Solow
Whether the reduction in comments reflects autonomous generation or overwhelmed reviewers, the signal is the same, the ratio of output to scrutiny is collapsing. In theory that would be acceptable if first-pass quality had risen proportionally. However the evidence, anecdotally at least, suggests the opposite. We have more, but not necessarily better. Output is accelerating yet realised value is not and measuring that gap precisely is difficult. Why do I think this is important? Because code is simply the most visible microcosm; the same dynamic is likely unfolding across documents, designs, reports, and decisions.
Back to the horses
A faster horse is still a horse. It moves more quickly, but it does not understand the terrain, the context of the journey, or where judgement is required along the way. Speed alone does not guarantee progress and in complex environments it can simply accelerate confusion. In a widely shared post3, AI thought leader Carl Engelmark described organisations investing heavily in generative assistants and internal agents, only to find that users still don’t fully trust the results. A common complaint being that the same question can yield different answers, and much of the time saved in generation is spent verifying whether the output is correct or even grounded in the right context.
“When an AI agent in your firm answers a question about “revenue recognition” or “material weakness” or “deferred consideration,” it’s drawing on a vast general understanding of those terms, trained on the world’s text, not your firm’s specific context, methodology, history, or judgment.”
Carl Engelmark
This is not a failure of effort, simply, it is the nature of stochastic systems. They optimise for plausible continuations, not domain-correct reasoning. In complex fields such as finance or audit, professionals report that while AI can automate components of a workflow efficiently, it still lacks the contextual judgement that experience provides.
The upshot is this – humans are not doing something fundamentally different. We are still interpreting, deciding and applying judgement. We have simply introduced a new actor into the system, one whose outputs appear coherent, but whose internal reasoning and basis of truth are no longer fully legible, and this actor can interact with everything.
“Program testing can be used to show the presence of bugs, but never to show their absence.”
Edsger W. Dijkstra
Defensibility Becomes the Product – The New Scarcity
For decades, frameworks like ISO 9001 insisted on something unfashionable (and boring) – that quality is not an act of brilliance, but a property of process. It is not whether something looks correct, but whether it can be shown to have been produced through controlled, documented, and repeatable steps. In slower eras, this could feel bureaucratic. In slower eras, this could feel bureaucratic. In an era of faster horses and stochastic generation, it may look like a millstone to some, yet to others it looks exactly like what we built it for.
“Beware of bugs in the above code; I have only proved it correct, not tried it.”
Donald Knuth
Why? Because output is now abundant, but assurance is not. The marginal cost of drafting, coding, designing, or analysing has collapsed; the marginal cost of verifying, contextualising, and defending those outputs has not. It is in this new gap that a new competitive frontier lies.
Organisations that can show why their artefacts exist, what they relied upon, and who stands behind them will earn trust. Those who can demonstrate the provenance and interoperability of outputs at the meta level will earn belief. To me this elevates the audit trail from a tick-box exercise, to the realm of actually being the asset.
“We had better be quite sure that the purpose put into the machine is the purpose which we really desire.”
Norbert Wiener
Defensibilty Becomes the Product – Legibility, Drift & Governance
Fluency is not the same as legibility. A model can produce an answer that reads confidently, references plausible sources, and mirrors domain language with uncanny precision. But fluency does not reveal lineage. It does not show which assumptions were embedded in the prompt, which data sources were weighted, or where contextual drift first entered the system.
The more we rely on that fluency, the greater the compounded risk. Where a human reviewer might skim or miss inconsistencies in 40 pages of requirements, dealing with them downstream with a smile and a small course correction, a machine incorporates every token and nuance into its output. Those inconsistencies propagate through delivery stages, magnified and transformed, and if not gatekept, silent. What looks coherent at the surface may carry latent errors inherited from upstream assumptions.
But, in fact, a bridge between the human mind and the machine mind is symbolic logic: mathematics. When we think clearly, we are intelligible to machines. People who write code know this: that the essence of making yourself clear to a machine is to think clearly yourself.
The machine has no patience for the half-truth, the analogy, the semi-grasped association. For the machine, everything has to be clear, everything must be defined.
Terrance McKenna4
Early drift is not a minor defect; it is compounding. By the time a misalignment is detected, outputs may be polished, approved, and operationalised. The process appears intact, but in reality, the foundation has been voided.
Anecdotally, just yesterday I was told that if something is missed in the early steps of an AI delivery pipeline, it is often better to delete everything produced post-mistake and rerun the process. The fingerprint of the mistake may be subtle, but it persists in assumptions and artefacts, propagated and morphed. While erasure may be acceptable during a very early stage or on an internal project, it does not scale to a running business.
And so governance re-enters the frame, not as ceremony, but as structural defence. Human oversight at stage gates, documented checkpoints, and independent validation loops maintain the integrity of delivery, and therefore of the contract, at regular intervals. Humans are critical; allowing an AI system to mark its own homework, using the same prompts and documents as those used to produce the homework, introduces circular assurance, self-referential confidence that is neither earned nor reliable.
Here, technologies like blockchain may reinforce governance by hardening trust assumptions. Immutable logs of prompts and revisions, cryptographic attestations of review, and timestamped approvals allow the machine to keep generating while keeping humans honest. Artefacts with verifiable lineage are strategically robust and may become a cornerstone of quality in the AI era. The lineage itself becomes defensibility.
“Nothing exists inherently. Everything exists in dependence upon conditions.”
Nagarjuna
Only as Much Chaos as You can Handle
Quality systems have always been chaos-limiting systems. They create structure, define expectations, and make human unpredictability manageable. In the era of AI, those predictable environments are now infused with controlled chaos, stochastic outputs, rapidly scaling volume, and outputs that look coherent but hide drift or latent errors.
The role of ISO-like frameworks is no longer just to ensure humans do what they said they would. Their new purpose is to define how much stochasticity can safely enter critical processes, and where governance, oversight, and traceability must act as buffers. Stage gates, checkpoints, and verifiable audit trails are the mechanisms by which chaos is made tolerable. Where ISO 9001 once guaranteed that humans could be imperfect inside predictable machines, the next iteration may guarantee that humans remain composed while navigating a world of imperfect, stochastic collaborators, both human and machine (or who knows, both?). I like the idea that the standard itself may become a measure of how much chaos an organisation can safely absorb and still deliver trustable outcomes, hey, it might even be fun. So, let’s give the horse, some wheels.
“A Discordian is Required… to use only as much chaos as he can handle.”
Principia Discordia
I intended to write some notes on how AI was both a super powering us and hobbling us at the same time, then realised how much existing assurance frameworks could support us in our activities and the post grew. If I fast forward a couple of years I imagine we will see AI auditors and emerging standards for the use of AI in different market segments – what is appropriate, and what less so. Right now I believe everyone wants the speed, and that we can all work at 20x, but that if we work at 16x we can also build quality first, garnering the love, respect and retention of our users and clients.
For Afters
A very short list of places where I think ISO 9001 could add new thinking:
Clause 7: Competence and Awareness: AI is unlike any tool humans have used before. New training and tailored approaches are needed to understand how to use AI responsibly, where it is appropriate in a workflow, and how humans and machines should interact. HR policies, role definitions & hiring guidelines, and accountability structures all need to reflect this new actor in the system.
Clause 8: Operation: AI is now a participant in every process. While the standard need not be prescriptive, auditors might place extra emphasis on 8.2 (ensuring requirements are met) and re-evaluate 8.3 (design and development) in AI-first organisations. Where AI generates outputs, process controls must explicitly account for verification, stage-gates, and drift management.
Clause 9: Performance Evaluation: A whole new audit domain is emerging. Traceability, lineage, prompt and data logging, and responsible use become auditable criteria. Humans cannot outsource judgement; AI-assisted outputs must be verifiable, reviewable, and accountable.
In short, a new, AI-first version of ISO 9001 could bake these considerations directly into the standard, embedding the use, documentation, and risks of AI into every clause.
- https://github.blog/news-insights/octoverse/octoverse-a-new-developer-joins-github-every-second-as-ai-leads-typescript-to-1/ ↩︎
- https://business.adobe.com/assets/pdfs/resources/sdk/discover-how-ai-is-powering-document-productivity/state-of-ai-in-documents.pdf ↩︎
- https://www.linkedin.com/pulse/your-ai-tools-smart-do-speak-accountancy-carl-engelmark-et6xe ↩︎
- https://www.organism.earth/library/document/psychedelics-in-the-age-of-intelligent-machines#:~:text=The%20machine%20has%20no%20patience,clear%2C%20everything%20must%20be%20defined. ↩︎