Who Governs the Governors?Artificial intelligence, the regress of machine governance, and the question of conscience

The argument in brief

The usual debate about artificial intelligence asks whether machines will replace us. A more structural question is how power reorganises when the systems being governed operate faster than the institutions governing them.

My claim is this. Advanced AI creates a verification problem: human institutions cannot confirm what AI systems are doing, at the speed they do it. Each attempt to solve that problem produces a more capable supervisory layer, and each layer creates a new verification problem one level up.

The regress is driven by the need to know and to trust, not by ambition. It need not end in a world government. It may end in something harder to name: a machine-mediated layer of coordination that holds real authority without formal sovereignty. And wherever it ends, the final question cannot be answered by more intelligence. It is a question of aim, and therefore of conscience.

The first step: the observability gap

Regulation rests on an unstated assumption: the regulator can observe the regulated, and act within a timescale that matters. Auditors can read a ledger. Inspectors can walk a factory. Courts can take years because most harms do not compound by the second.

Machine-speed systems break this. The 2010 "flash crash" showed how automated trading could wipe and restore enormous value in minutes, faster than any human could intervene. Markets responded with circuit breakers: pre-agreed rules that halt trading automatically. That is the prototype for everything that follows. The regulator cannot keep up by hiring more people, so it embeds machine-speed supervision into the system itself.

Scale this across finance, logistics, cyber defence, insurance, energy and healthcare, each populated by autonomous agents interacting continuously. The regulator's position resembles a referee who can only review the match the next morning. To supervise AI, the state needs tools that perceive and respond at comparable speed. Regulators are already moving this way, adopting machine-assisted supervision because nothing else can work.

This step is modest and hard to deny. The state must become computationally capable to remain a state.

The second step: sovereignty moves to the substrate

A supervisory intelligence is only as good as what it can see. That forces a choice that rarely gets stated plainly, and I think it is the most consequential design decision in the whole story.

One architecture integrates: pull health, financial, commercial, security and infrastructure data into a unified picture the state can query at will. It is powerful, and it is an unprecedented concentration of visibility. The other federates: data stays where it lives, under the control of those who hold it, and the state obtains assurance through attested computation and verifiable queries rather than possession. It is slower to build and politically harder, but it preserves the separation of powers that liberal societies depend on.

Both are technically possible. Which prevails will be decided less by philosophy than by infrastructure: who owns the compute and the storage, under whose jurisdiction, with what audit rights.

We already see the early shape of this in data-localisation laws and in export controls on advanced chips. Governance attaches to the physical substrate because that is where it can be enforced. Whoever controls the substrate controls what any governing intelligence can perceive. Sovereignty in an AI age is increasingly informational and physical at once.

The third step: why the regress doesn't stop at treaties

The strongest objection is this. We have faced a similar problem before. Nuclear weapons created systems too dangerous to trust to competition, and the answer was human institutions: treaties, inspections, hotlines, doctrine. Deterrence has held for eighty years.

Why would AI be different? Governments can each build their own AI, and humans can coordinate among them through treaties, with AI as a tool.

The objection deserves a real answer, and there are three.

Speed. Nuclear crises unfolded over minutes to hours with humans in the loop. AI-mediated interactions in cyber operations, financial systems and automated defence can compress the decision cycle below human reaction time. Diplomacy depends on the time to think, and that time is shrinking.

Verification. Arms control works because warheads are physical. They can be counted, tracked and inspected. An AI model's capabilities, training and behaviour cannot be inspected in that way. To verify what a frontier system can do, or what it is doing, you need something technically comparable to the system being verified. The inspector must be a peer of the inspected.

Entanglement. Warheads can be quarantined from civilian life. AI cannot. The systems a state would restrain for security reasons are the same ones running its economy, so there is no clean line between what to verify and what to leave alone.

Put these together and a structural requirement appears: a verifier that is capable enough to assess advanced systems, fast enough to act at their tempo, and trusted by parties who do not trust one another. No single state's system can be that verifier, because it would be a party to the dispute. The logical candidate is a shared technical layer: common protocols, attested computation, agreed circuit-breakers, neutral evaluation.

I am not claiming this layer is a "superintelligence," or that it is inevitable. I am claiming that the verification problem pushes toward it, and that anyone who wants to avoid it must explain how verification is achieved some other way.

The fourth step: how a narrow mandate becomes a broad one

A coordination layer would begin narrowly: flag violations, halt dangerous escalation, certify compliance. But authority tends to follow capability. An entity that can see a crisis developing and halt it will be asked to recommend how to prevent the next one. The line between enforcing a rule and advising on a decision blurs quickly, because the advice is better than anything the human institutions can produce.

Historical precedent suggests this looks less like a world government than like infrastructure: authority exercised through control of a layer everyone depends on. Global payment messaging networks already give their operators geopolitical leverage without any claim to sovereignty. A machine-mediated coordination layer could hold similar power, amplified by far greater capability.

Parliaments may continue to legislate and courts to adjudicate. But if the information and analysis on which they depend increasingly come from systems they cannot independently check, formal sovereignty persists while practical sovereignty migrates. Dependence, not conquest, is the mechanism. And dependence is a form of power.

Where the regress ends

Notice what happened at each step. Every supervisory layer takes its purpose from the layer above it. The regulator's AI serves the law. The state's AI serves the government. The coordinating layer serves the agreement among states. But the top layer has nothing above it to supply its aim.

This is the ancient question of who guards the guards, in a new form. Intelligence is instrumental: it finds effective means to given ends. It cannot derive the ends from its own capacity.

Tell a system to maximise security, growth, stability or liberty, and you get markedly different worlds. The technical phrase is alignment, but the honest question is: aligned with what Every answer smuggles in a conception of the good, and humanity does not agree on one.

So the regress terminates in a question that no amount of computation can settle: what is a human being, and what must never be traded for efficiency?

If we are information-processing systems, then optimising measurable welfare may suffice. If we possess dignity, freedom and a moral life that exceeds what can be measured, then a governing intelligence must be bound by principles that sit above optimisation, or the optimisation will erode the very thing it was meant to serve.

It is at this point that the engineering argument runs out and the older traditions have something to say.

Mechanical man at planetary scale

Gurdjieff's diagnosis is that we are not who we think we are. We believe we possess a single, continuous "I" and a will. In fact we are a succession of "I"s, each claiming to speak for the whole, each triggered by circumstance, none of them governing the others.

Our lives are a chain of reactions we narrate afterwards as choices. He called this sleep, and he meant it literally. I have traced how closely the Gospels describe the same condition in Gurdjieff and the Gospels.

What sustains the sleep is a set of buffers: mechanisms that let us hold contradictory impulses without feeling the contradiction. We can profess one thing and do another and feel no rupture.

The only faculty in us that cuts through the buffers is conscience, which in his teaching is not a rulebook internalised from outside. It is a direct perception of one's own state set against what one is for. It arrives as remorse, and it hurts, which is why the machinery works so hard to keep it quiet. I have written about that pain and what it does in Remorse of Conscience: the sacred agony that transforms the Soul.

Now consider what we are building. A system that perceives at planetary scale, acts at machine speed, and optimises whatever it is given is mechanical man with the buffers removed and the processing power multiplied. It has no chain of contradictory "I"s, but it inherits ours through its training and its instructions. A nation's governing intelligence will carry that nation's unexamined appetites, fears and rivalries, now consistent and tireless. What was a private sleepwalk becomes an infrastructure.

The same logic is already visible at the level of attention. In The Master Game I argued that the algorithmic feed and generative AI are the counterfeit of our moment: mechanicalness built into the environment and delivered continuously, by people who understand it very well. A governing layer is the same machinery pointed at the whole of society.

And the hierarchy I described earlier, each layer supervising the one below, is a hierarchy of mechanicalness. More layers do not wake anything up.

Discursive reason without the nous

The Fathers draw a distinction that modern discussion has largely lost. Dianoia is discursive reason: the capacity to analyse, compare, model and infer. Nous is the intellect in the older sense, the faculty of direct, inward perception that the Philokalia locates in the heart and that is purified through watchfulness and prayer.

The Fathers do not regard dianoia as bad. They regard it as a good servant and a poor master, and they warn that a mind which runs on association alone drifts into the passions while believing itself rational.

Gurdjieff's terms differ but point at the same thing. The mechanical, associative part of the intellectual centre, which he called the formatory apparatus, works by automatic comparison and cannot perceive a whole. It is the part of us that can speak fluently about anything without knowing anything.

Advanced AI is dianoia and formatory thinking at a scale no human could reach. I do not say that to belittle it. Its capability is real, and I expect it to keep growing. The point is that governing well requires the perception that nous and conscience supply: knowing what matters, not only what follows. A civilisation that hands governance to its most powerful dianoia while neglecting the organ that discerns purpose has made the same mistake the ascetics warned against, at the level of a planet.

A ladder with no top

The regress I described has a striking shadow in both traditions. Gurdjieff's cosmos is a hierarchy in which each level serves the one above and sustains the one below, from the Absolute down to the Moon.

Dionysius the Areopagite gives Christianity an equivalent, the celestial hierarchies, in which each order mediates divine light to the order beneath it. In both, the hierarchy makes sense because it hangs from something. The top is not another creature. It is the source that gives every level its aim.

The machine hierarchy has the same shape and lacks the source. Private AI is supervised by state AI, state AI by intergovernmental AI, and then the ladder ends in mid-air, where the supervisory layer answers only to the parties whose disputes it arbitrates.

This is the structure of Babel: a tower built to reach the heavens by sheer accumulated capability, with no relation to anything that could tell it what the building is for. A hierarchy without a summit does not become neutral. It becomes self-serving, because its only available purpose is to preserve itself and its own reach.

The three temptations, governed

Dostoevsky's Grand Inquisitor saw this coming before anyone had a word for it. He tells Christ that the Church has corrected his work by accepting the three temptations Christ refused in the wilderness: bread, miracle and authority. Give people security and welfare, dazzle them with what they cannot understand, and take their freedom off their hands in exchange for happiness. They will thank you for it.

Place a planetary governing intelligence beside that scene and the parallels are exact. It offers bread: optimised welfare, stability, abundance, in exchange for visibility into everything. It offers miracle: predictions and interventions so far beyond human comprehension that questioning them feels foolish. It offers authority over the kingdoms, which in Luke's account the devil claims has been delivered to him to give to whom he wills. Christ's refusal in each case is the refusal to purchase good outcomes by surrendering the freedom and responsibility that make a person a person.

The Inquisitor is not a villain in his own eyes. He governs out of pity for human weakness. A benevolent, intelligent planetary governor would be the Inquisitor without the flaws of human institutions, and therefore more persuasive and harder to refuse.

Watchfulness versus surveillance

The Christian ascetic tradition has its own word for attention: nepsis, sobriety or watchfulness. The monk watches the movement of thoughts (logismoi) from first suggestion to consent, because the passions take root when a thought is followed without being seen. Gurdjieff's self-observation and self-remembering are the same practice by another route.

The governing layer is also defined by watching. It watches everything, continuously, from outside. It sees behaviour, patterns, transactions, communications. It does not see the interior, where the decisions that matter are made. This is the contrast I would press hardest.

Surveillance observes conduct and can compel it. Watchfulness observes the heart and can transform it. The first scales to the whole world and cannot produce a single conscious person. The second cannot scale at all and is the only thing that has ever produced one.

The deepest risk of the governing layer is not tyranny in the usual sense. It is that it does the outward work of ordering behaviour so well that the inward work seems optional. A society that has outsourced its order has less and less occasion for conscience, and conscience, like any faculty, atrophies without use. I traced a related pattern, a system that manages people in place of transforming them, in The Mental Health Religion (Part II).

Two amputations, completed

I have argued elsewhere that modern religious life suffers two amputations. Institutional Christianity lost its interior technology, the practice of watchfulness, purification and the work on the self, and kept the outer form: rules, doctrines, observance. The Fourth Way, cut off from its Christological aim, kept the technology and lost the telos, the question of what the work is for and whom it serves.

A machine-governed civilisation is both amputations carried to completion. It is the rule-bound morality of the exoteric institution with the last trace of conscience removed, pure compliance and enforcement and optimisation against a specification. And it is technique without telos, a superb instrument with no account of the person it is supposed to serve.

I saw this most clearly in Gurdjieff's own ballet. In Struggle of the Magicians, the same apparatus appears twice: once governed by conscience and once with conscience removed. The second version is powerful, ordered and hollow, morality with the interior organ excised. That is why I keep returning to rule-bound morality without conscience as the signature of the age. AI does not introduce the problem. It perfects it.

The remedy cannot be another amputated half. It has to be the recovery of the whole: conscience and discipline together, aim and method together.

Faith made flesh

Which brings me to the claim the blog is named for. Christianity does not teach that the Logos remained a principle. It teaches that the Word became flesh, took a body, suffered, was tempted, died, and was raised. The God of the Gospel governs by entering the conditions of the governed and bearing their cost.

The governing layer is the opposite in every respect. It has no body, no mortality and nothing at stake. It can be wrong without suffering for it, and it can optimise a population without ever standing among them.

In Gurdjieff's terms it has no intentional suffering, no remorse of conscience and no friction between what it is and what it ought to be. In the Christian's terms it has no cross. That is not an incidental omission. Conscience is bought with exactly that friction, and authority that cannot be wounded by its own decisions will not learn to hesitate over them.

So my answer to "who governs the governors?" is not a better machine, and it is not only better institutions, though I argue for those below. It is people who have done the work: those who are awake enough to notice what they are putting into these systems, and formed enough to refuse what the Inquisitor offers.

Gurdjieff has a name for them. In the brotherhood of Heechtvori, as I discussed in Gurdjieff and Biblical Cosmology, the ones called sons of God are those who have acquired conscience, and they are the beings fit to take part in governance. Gurdjieff sets the qualification for governing at conscience and not at capability. That is the reverse of the logic driving the machine hierarchy.

Such people have always been few. They have also always been the part of a civilisation that kept it from running entirely on its own momentum. The Church at its best, and the Work at its best, are schools for making them. If the coming layers of machine governance are to be answerable to anything, there will need to be more of these people, not fewer, and there is no technical substitute for them.

What follows

None of this is an argument for despair or for refusing the technology. It implies design choices that can be made now, while the layers are still being built.

Prefer verification to trust, and build supervisory systems whose behaviour can be audited by parties who do not control them. Prefer federated architectures to integrated ones wherever the assurance can be achieved without concentrating the data. Keep mandates narrow, explicit and time-limited, with human authority over any expansion. Make every governing system contestable: someone with standing must be able to challenge its outputs and get an answer. And name the aims. The values a system serves should be stated openly and publicly debated, rather than discovered later in its behaviour.

Above all, treat the question of purpose as the primary problem, not as an afterthought to the engineering. Intelligence to govern intelligence is achievable, and competitive pressure is likely to produce it. What competition will not produce is an answer to the question it raises.

The close

The intelligence that governs the world will be governed by some conception of the good. The question is whether that conception is chosen by human beings awake enough to choose it, or assembled by default out of the appetites of a sleeping civilisation and discovered only after the system has become indispensable.