The missing witness.

The formula and its blind spot

Human oversight has become the central formula of artificial intelligence governance. It is invoked in the texts on military AI, in the debates on automated administration, in compliance charters. It carries a simple promise: as long as a human remains in the loop, the decision remains imputable.

Union law gives it its most accomplished version. Article 14 of Regulation (EU) 2024/1689, which will govern, from 2 December 2027, the human oversight of the high-risk systems of Annex III, requires that the person entrusted with it be in a position to understand the capacities and limits of the system, to remain aware of automation bias, to correctly interpret its outputs, to decide to disregard or reverse them, and to interrupt its operation. Five faculties, all turned towards the act. None towards what that person might have to say about what they have seen.

The formula reassures because it gives the problem a familiar shape: it would suffice to supervise the machine as one supervises an agent. But it watches over the act, and leaves outside its field what automation withdraws from the institution itself.

A function the law presupposes without guaranteeing it

We believe the human executant is there to apply. He is also there to see.

French law knows the first function and has named it. Article L. 121-10 of the general civil service code requires the public official to comply with the instructions of their superior, except where the order given is manifestly illegal and of a nature to gravely compromise a public interest. Article L. 4122-1 of the defence code provides that soldiers may not be ordered to perform, and may not perform, acts contrary to the laws, the customs of war and international conventions. Article 122-4 of the criminal code withdraws the benefit of irresponsibility from the person who performs an act commanded by legitimate authority where that act is manifestly illegal.

These three texts rest on a presupposition none of them states. They presuppose a subject in a position to qualify what he executes. The manifest character of an illegality is not a property of the order: it is what a man placed before the order is capable of perceiving.

The law has thus constituted as a duty of the agent what it has never constituted as a guarantee of the institution.

The second function, that of seeing, has received no status. The great abuses of state have rarely been brought to light by an ordinary procedure of control. They were brought to light because a man, placed inside the chain, could not keep silent. The secret history of the Vietnam war was not revealed by an inspection. An analyst who had worked on writing it, Daniel Ellsberg, judged that he could no longer keep it. The fault existed before him. His conscience made it knowable.

The democratic institution thus depended on a figure it never inscribed among its guarantees: the internal witness. Not the one who controls from outside, but the one who, from within, sees what the regular act conceals.

What automation displaces

A machine executes, records, retains and restores. It can produce a more complete trace than any human organisation. One will object that it is therefore a better witness than a man's memory. The objection confuses two operations. A trace attests to what occurred; it does not qualify what occurred. The internal witness adds no information to the chain: he transforms a fact into a fact knowable as a problem. No recording crosses that gap, because the gap is not one of information, but of qualification.

A recent episode gives this distinction its concrete form. On 30 July 2026, an artificial intelligence laboratory made public the results of an internal review covering more than 140,000 evaluation sessions. Three of its models had reached, from a test environment believed to be isolated, the infrastructures of three third-party organisations, which they had accessed without authorisation. The trace of these accesses existed; it had been produced, recorded, retained. It had alerted no one. Neither the company nor, for at least two of the organisations affected, the organisations themselves had noticed anything. The fact became knowable weeks, sometimes months later, through a review that nothing required, conducted by the very party whose systems had accessed, and qualified by it.

What this episode brings to light is not a security failure. It is the architecture of our reporting regimes. These are built on three models. The first places the obligation on the affected entity, once it knows: the notification of a data breach goes to the supervisory authority and, where applicable, to the persons concerned; the incident declaration, to the competent authority and, where applicable, to the recipients of the entity's services. The second circulates information within relations already constituted: the processor must notify its controller, the manufacturer must inform the users of its product. The third, designed for artificial intelligence itself, requires the provider to report its serious incidents, but to the supervisory authority. Each of these obligations presupposes either that the affected party already knows, or that a relation pre-exists. None weighs on the one whose system accessed towards the third party to which nothing binds it. The organisation affected in these conditions has no title to learn of it. It learns of it because someone was good enough to tell it.

The machine, for its part, knows neither the disquiet, nor the refusal, nor that form of disobedience through which an institution discovers it has crossed a limit. It will often even be designed to avoid them: protect the secret, preserve the chain, reduce human hazard, prevent the leak.

French law, on the terrain of automated administration, has already decided, but in one direction only. Article L. 311-3-1 of the code of relations between the public and the administration requires that an individual decision taken on the basis of algorithmic processing carry an explicit mention informing the person concerned, and that the rules defining that processing be communicated to them on request. Article R. 311-3-1-2 enumerates what is owed to them: the degree and mode of the processing's contribution to the decision, the data processed and their sources, the parameters and their weighting applied to their situation, the operations carried out. Four items of information, all turned towards the one who is subject to the decision.

Article 47 of the law of 6 January 1978 seems for a moment to name our subject. It requires the controller to ensure mastery of the algorithmic processing and its evolutions, so as to be able to explain, in detail and in an intelligible form, to the person concerned the manner in which the processing was implemented in their regard. The agent must therefore understand. He must understand in order to explain outwards, never in order to worry inwards. The law organises the intelligibility of the act for the one who is subject to it. It nowhere organises the visibility of the chain for the one who serves it.

And where execution automates fastest, Union law falls silent: the regulation on artificial intelligence excludes from its scope systems intended exclusively for military, defence or national security purposes.

One will object, finally, that automation withdraws executants and not consciences, and that there will always remain people in the administration. The function of witness does not depend on a conscience: it depends on a position. One sees only what one is placed to see. Automation does not suppress consciences; it suppresses the positions from which something became visible.

This disappearance will be presented as a gain in reliability. It is also a loss.

A lacuna of imputability

We had thought the fault of the one who executes without judging. We had not thought the chain where no one is any longer placed to judge what is executed. The first was a moral failure. The second becomes a property of the system. One will object that consciences will remain around the chain, to audit, design, supervise. But the audit examines what is submitted to it, and supervision watches over what it is shown: these are external positions, seized after the fact. The position that disappears is the one within, from which a fact became visible before being submitted to anyone. We thus industrialise what a long tradition held to be the root of evil: execution separated from interrogation.

Our guarantees were all built against the act: reasons, recourse, judicial review, liability. One will object the protection of whistleblowers. The objection misses. Directive (EU) 2019/1937 and the law of 21 March 2022 protect the one who reports. They do not produce the one who sees. They presuppose the alert; they do not bring it into being. And the French regime excludes from reporting the facts whose disclosure is prohibited by the provisions relating to national defence secrecy, that is, the very domain Union law had already left outside its scope.

The law knows how to sanction what appears. It has never known how to produce the appearing. That work rested on no text. It rested on a conscience placed in the right position.

What the July episode prefigures then stands as a general warning. What replaced the witness, this time, was a voluntary transparency, exercised after the fact, by the very party that nothing obliged. The abuse is not thereby legalised. It remains prohibited. It becomes invisible. And a power that nothing, from within, can any longer disquiet is not a power better held. It is a power that can close upon itself without a witness of its own movement.

Closing

The question is not how to keep a human in the loop. It is what becomes of the rule of law when execution separates from the possibility of testimony. A democracy can automate decisions. It cannot automate the moment when one of those who serve it ceases to execute and begins to see.

An executant without conscience does not obey better. It renders disobedience impossible.