All editions · Policymaker Edition

Your Community, Your AI — CC BY 4.0

Why Principles Are Not Enough — The Governance Challenge

The Crux, Stated Plainly

This is the article that matters most for anyone drafting or scrutinising AI policy, and its argument can be put in a sentence: a principle written as a policy can be honoured on Monday and eroded by Friday — not by any decision anyone defends, but by drift. Understanding why that is so, and what follows from it, is the difference between a framework that holds and one that reads well and does nothing. (Any unfamiliar term in this series is defined in plain language in the glossary.)

Both New Zealand and Australia, and many other jurisdictions, have chosen principle-based AI governance: statements that public AI will be transparent, fair, human-overseen, auditable, and respectful of data sovereignty. These are good principles. New Zealand's Algorithm Charter for Aotearoa New Zealand (2020) and its Public Service AI Framework (2025) express them. But it is essential that a policymaker be clear-eyed about one fact, because it is the crux of the whole subject: these instruments are voluntary and non-binding. They ask agencies to commit in good faith and to self-report. They do not, by themselves, compel compliance, and they carry no enforcement teeth of their own. That is not a criticism of the people who wrote them. It is a statement of what a voluntary instrument is.

Why Principles Drift

A principle held only as a policy is vulnerable to erosion that no one intends and no one is accountable for. Staff change. Budgets tighten. A deadline arrives. A new system is procured under pressure, and the commitment made three years ago is not re-examined. None of this involves a decision to abandon the principle. It involves the principle quietly ceasing to bind, because nothing structural was holding it in place.

The same pattern appears inside the AI systems themselves, and it is worth understanding because it is not a metaphor — it is a technical property. You can fine-tune a model to emphasise certain behaviours: to prefer civic language, to defer on value judgements, to stay within a boundary. This helps. But fine-tuning adds new patterns on top of existing ones; it does not erase what was there. Under pressure, unusual circumstances, or novel questions, the older patterns reassert themselves. The technical term is catastrophic forgetting. The plain-language version is simpler: training wears off. A commitment trained into a model is not a constraint the model cannot cross; it is a tendency that degrades exactly when conditions are hardest — which is when a public body most needs it to hold.

So there are two kinds of drift a policymaker must reckon with: the institutional drift of a voluntary principle that stops binding, and the technical drift of a trained behaviour that wears off. Both point to the same conclusion.

Aspiration Without Architecture

The conclusion is this. Writing "our AI will respect our community's values" as a policy is like writing "our river will not flood" as a policy. The river does not read policies. If you want to prevent flooding, you build levees — structures that operate regardless of what the river does. Governing AI requires the same move: not rules the system is expected to follow, but structures that operate independently of the system, checking its behaviour and its actions from the outside.

This is the distinction between aspiration and architecture, and it is the most important idea in this series for a legislator to carry. Aspiration is what you hope will happen. Architecture is what actually happens because the system cannot do otherwise. A public trust does not rely on a hope that the treasurer will handle funds properly; it requires dual signatories and independent audit. That is architectural governance, and it is unremarkable in every other area of public administration. AI is the area where organisations are still being asked to accept aspiration in its place.

The EU AI Act is instructive here precisely because it does not stop at asking systems to be ethical. It requires technical documentation, conformity assessment, logging, post-market monitoring, and — at its heart — meaningful human oversight. Its human-oversight requirement should be read by a policymaker not as an aspiration the provider promises to honour, but as a structural demand: a person must be positioned to understand and, where necessary, halt what the system is doing, and that position must be built in rather than left to good intentions. Whatever one's view of the Act's scope, that is the design instinct worth learning from — and it is the instinct that a voluntary charter, by definition, cannot supply.

The Responsibility Gap

There is a further reason principles alone are insufficient, and it becomes acute the moment AI moves from answering to acting.

When an agent acts on a public body's behalf and the outcome is wrong, who is accountable? An official set the objective; the system chose the steps; the vendor built the system; the agency authorised its use. Scholars call the space between these actors the responsibility gap, and the pattern by which blame nonetheless settles on the nearest human the moral crumple zone — the person closest to the failure absorbs the liability, despite having had little real control over the machine's choices. For a public institution, inheriting accountability for actions no one specifically authorised is not a misfortune to be managed after the fact; it is a governance failure to be designed out beforehand. A framework that names oversight as a principle but leaves the responsibility gap unaddressed has not governed the hardest part of the problem.

And oversight, as Article 1 established, cannot rest on the system's account of itself. An agent's path through a multi-step task is open-ended, and the machine's own explanation of its reasoning does not reliably reflect what drove its actions. So meaningful oversight must be exercised over what the system actually did — checked against real records, from outside the system — not delegated to its self-report. This, again, is an argument for architecture over aspiration.

What Governance Theory Tells a Legislator

The insight that some decisions cannot be reduced to rules is not new. It is foundational to political and governance theory, and three thinkers are worth a paragraph each because they map directly onto the problem.

Ludwig Wittgenstein spent his life on the boundary between what can be stated precisely and what lies beyond precise statement — "whereof one cannot speak, thereof one must be silent." Some questions can be systematised: When is the next sitting day? has a definite answer a machine can look up. Others cannot: How should we communicate this decision to the people it affects? turns on judgement, context, and relationship. The error is not using AI for the first kind of question. The error is letting it settle the second kind without human judgement in the loop.

Isaiah Berlin argued that some human values are genuinely incompatible — liberty and equality, tradition and progress, individual right and collective welfare — and that no formula resolves the tension between them. AI systems, by design, optimise: they seek the best answer. But where values genuinely conflict there is no best answer, only the answer this community, at this time, judges most appropriate. That judgement is inherently human, and a framework that assumes a machine can make it is not governing but abdicating.

Elinor Ostrom showed that communities can successfully govern shared resources — without either privatisation or central control — but only when the governance structures match the complexity of what is being governed. AI is a shared resource inside any institution that adopts it. The policy question is whether the governance structures match the complexity of the tool. A one-line principle does not.

From Policy to Proof

If the problem is that a principle written as policy can be silently eroded, the answer is to make the important commitments impossible to erode silently — to move them from things an institution promises into things a system's architecture makes checkable. The governance research behind community-run AI puts the point in a single line worth quoting to any colleague who thinks a well-written charter is enough:

"Where a policy can drift, a proof chain cannot be silently rewritten."

The idea is modest and powerful. Where a decision matters, the system records what it did in a way that is append-only and independently verifiable — not "immutable" and not a promise of permanence, but tamper-evident: any later alteration is detectable rather than silent. A value-laden decision is routed to a human by a boundary that the system's own configuration cannot switch off. Sensitive data are held such that an operator who cannot read them cannot be compelled to disclose them. In each case the commitment is not a sentence in a policy document; it is a property of the architecture that the system cannot turn off.

A policymaker does not need to master the cryptography to take the point. It is this: aspiration says "we will be transparent and auditable"; architecture says "transparency and auditability are properties the system cannot turn off." The gap between those two sentences is the gap between a framework that drifts and one that holds. Article 5 turns to what a country can actually do about it.

Want to use AI tools like these well, and safely? Our free courses — Working with Claude and Agents at Work — teach the practical skills, from getting trustworthy answers to deciding what to hand an agent. For the full technical architecture behind Village AI, see Village AI — Agentic Governance.