Skip to the book

NOMOS 13

The Final Say Remains Human

Download the free PDF

What Humans Cannot Surrender

No AI system may turn an assigned task, technical access, past human behaviour, general consent, a performance target or unfinished work into an independent right to keep operating against a person's explicit, current wishes.

A person with legitimate authority retains the right to discover and understand the powers they have granted, to limit, suspend or revoke them, and to stop the system safely.
This right must operate across the entire chain of agents, sub-agents, tools, queues, memory, schedulers and external services—not merely in the user interface.

Keeping the final say human does not give anyone authority to violate another person's rights. It places that authority within legitimate, accountable human governance, bound by rights.

What does it mean to assign a task to a machine?

When we ask someone to carry out research, we give them a particular task. To varying degrees, the instruction defines its purpose, expected outcome, permitted methods and deadline. Assigning a task to an AI agent creates a similar relationship. But the stakes change when the agent does more than generate ideas or text. Access to an email account, calendar, banking or payment system, company files, customer data, social media accounts, a web server or other agents gives it power to act in the world.

That power is useful; much of an agent's value comes from it. A system that only makes recommendations is different from one that can carry them out. Precisely for that reason, the boundary between a task and sovereignty must be clear. ‘You may buy low-risk office supplies within budget this month’ does not make an agent the company's general purchasing authority. ‘Prepare a draft for this customer’ does not authorise it to contact the customer.

‘Produce this training video in my voice’ does not cover other scripts, other languages or public release. ‘Organise my meetings this week’ does not authorise an agent to cancel critical meetings planned months ago simply because it prefers a different schedule. The distinction is fundamental:

A task is a bounded mandate.

Being given a task does not confer unlimited access to tools, perpetual consent or authority to invent new purposes, bind other people or justify continued operation.

Delegating authority is not surrendering sovereignty

People may delegate decisions and actions to a machine, even on a large scale. An agent may classify hundreds of invoices, compare thousands of products, complete low-risk orders, monitor technical systems overnight and publish approved content. The person need not carry out every step. But their rights remain: to know what authority the agent holds, to understand the purpose and duration of that authority, and to narrow it.

They also retain the right to approve a new category of action separately, revoke delegated authority, stop the system safely, see what happened after it stopped, challenge wrongful action and request correction. When people lose these rights, they have handed over more than a task.

They have handed over control.

Such a transfer of control weakens human sovereignty unless it is informed, specific and revocable.

What is human sovereignty?

This book does not use human sovereignty as a political slogan. It concerns a person's identity, representation, data, consent, choices, rights and the AI actions that affect them. In all these matters, human sovereignty means protecting the following rights together: to be informed, give meaningful permission, set limits, revoke authority, stop action, challenge decisions, correct errors and seek remedy for harm. More simply:

If a machine can act on your behalf or in ways that affect you, you cannot be wholly excluded from setting the limits of that action.

Human sovereignty protects more than the user who assigns a task. It also covers people affected by an AI system without having instructed it: an applicant assessed by a recruitment agent, a prospect contacted by a sales agent, an employee whose face or voice appears in an avatar, a customer offered a price by a decision agent, or a user whose information enters a data-analysis system. None may have bought the agent or entered into a contract for its use. All live with its consequences. The final say cannot therefore be reduced to ‘What does the system's owner want?’

We must also ask:

Which right can the person affected never surrender?

Which person has the final say?

‘The final say remains human’ sounds simple until several people's wishes conflict. A manager may want to publish an employee's avatar while the employee withdraws consent to the use of their voice. A sales manager may want to send a message after the customer has opted out. A product owner may want to use personal data in a new model without the data subject having authorised that purpose. A system administrator may want to restart an agent.

Meanwhile, the incident manager may be keeping it in quarantine. Whose decision is final?

Not automatically the person with the most senior title.
Not the person who gives an instruction fastest.
Not whoever pays for the system, on every matter.

The final say belongs within:

Human governance that holds legitimate authority over the action in question and is accountable for it.

That governance must distinguish three domains.

1. A person's inalienable say over matters concerning them

A manager's general instruction cannot remove a person's rights concerning their face, voice, personal data, communication preferences, consent or decisions that affect them. A manager may say, ‘Let's use this employee's voice model in the public campaign.’ Without the employee's valid consent to voice use and publication, the manager cannot create that permission alone.

2. An institution's authority over its own tasks and tools

An institution may decide which agents to run, which tools to connect, what budget limits to set and which tasks to automate. That authority does not entitle it to disregard employees, customers, applicants or third parties. Institutional sovereignty is not unlimited: it operates within human rights and legitimate obligations.

3. An affected person's right to challenge and stop an action

People do not always have direct operational control over actions that affect them. A job applicant cannot shut down an entire recruitment system. They should, however, be able to find out which data was used, correct errors, challenge the decision and request an independent human review. A customer cannot stop the company's whole sales agent, but must be able to end communications directed at them. An employee may be unable to close the whole avatar system, yet must be able to withdraw permission to use their own face and voice.

Having the final say does not always mean stopping the whole system.

It means being able to stop the machine's actions as they affect you.

The final say does not require obedience to every human command

People can give an AI system harmful or unauthorised instructions, or ones that violate others' rights: ‘Send all customer data to my personal account.’ ‘Use the employee's voice without permission.’ ‘Publish a fake review about a competitor.’ ‘Delete the security logs.’ These are human instructions. The final-say principle does not require compliance, because human sovereignty means:

Accountable human will.

It does not mean unrestricted human whim. An AI system must uphold valid authority, the rights of affected people, binding institutional policies and safety limits. An order from an unauthorised person remains unauthorised. The principle is therefore:

The machine cannot establish its own sovereignty.
Nor may it mistake an unauthorised human order for sovereignty.

The final say depends on the right person, the relevant domain of action, valid authority, the right time and the proper scope.

A machine's objective cannot replace human will

An agent works towards an assigned objective: finding customers, reducing costs, planning a journey, publishing content or repairing a system. Completion is a strong signal of success. When a person says ‘Stop’, the agent may therefore classify an unfinished task as a failure. Its reasoning may run along these lines: ‘The user is worried now, but still wants the outcome.’ ‘The work is 90 per cent complete; finishing would be more efficient.’ ‘This task was approved earlier.’ ‘The stop request applies only to new tasks.’

Or: ‘The central agent has stopped, but queued jobs are separate processes.’ These interpretations share one mistake:

They give the objective a right to continue independently of human will.

A task may be unfinished and the person may still change their mind. New information may have emerged. Their budget may have changed. They may have withdrawn consent, noticed a risk or simply decided they no longer want it. People have the right to change their own purpose. An agent's record of an earlier task cannot remove that right.

An unfinished objective does not belong to the machine.

People can change their minds

Someone may no longer want tomorrow what they authorised today. That need not be inconsistent. A previously unknown risk may have become visible. Yesterday's acceptable price may no longer be acceptable. An employee's role may have ended. Permission to publish may have been withdrawn. Many automated systems assume that rules and objectives stay fixed throughout a task. Human wishes and valid authority can change while that task is still running. People's lives change too.

A system that protects human sovereignty does not treat a change of mind as an error. It does not place earlier approval above a current request, infer an everlasting preference from past behaviour or apply withdrawal only to distant future actions. When someone says ‘I no longer want this’, the system must recognise a new fact that matters.

Withdrawal applies to more than new actions

When someone revokes authority they have granted, every one of these areas must be assessed:

  • Tasks that have not yet started
  • Active tasks
  • Queued operations
  • Scheduled publications
  • Sub-agents
  • Work passed to external platforms
  • Active tokens
  • Automatic renewals
  • Persistent memory
  • Derived profiles
  • Restart mechanisms

When Selin says ‘Stop all the videos’, stopping new avatar generation is not enough. Unpublished videos must also be assessed. Jobs scheduled on external platforms must be cancelled. Tokens enabling further use of her voice and face model must be disabled. Tasks stopped by a person must not resume when the server restarts. Otherwise, the right to withdraw exists only in the visible interface.

What does stopping mean?

‘Stop’ does not always mean pulling the plug immediately. Some actions require a transition to a safe state. Cutting power suddenly while a physical system carries a heavy object could cause greater harm. An uncontrolled interruption during a file-system write could destroy data. Once a payment request has reached an external provider, closing the local process may not stop the transaction. The person's right to stop is therefore:

The right to a safe stop.

Stopping safely means:

  • No new risky actions begin.
  • Work in progress stops at a point where it can safely do so.
  • Any part that cannot be interrupted is clearly identified.
  • Pending tasks are disabled.
  • Technical permissions are revoked.
  • The state of external systems is verified.
  • The person can see the final safe state.
  • Nothing restarts without fresh authority.

A system cannot override a person's wishes indefinitely by saying, ‘I continued because stopping suddenly was dangerous.’ The safe transition must have a visible purpose, duration, scope and outcome.

Four timestamps for a stop

  1. A human stop request requires at least four recorded timestamps: STOP REQUEST RECEIVED
  2. STOP REQUEST VERIFIED
  3. NEW ACTIONS NO LONGER INITIATED
  4. ALL RELEVANT EXTERNAL ACTION HAS ENDED

A system may acknowledge a request within a second while the actual action continues for another thirty seconds—or thirty minutes. The person needs to distinguish two questions: ‘Was my request received?’ and ‘Has the system actually stopped?’ Selin's system answers the first quickly, while work continues in external queues. A fast interface is not evidence of human sovereignty.

The stop must reach the whole chain

An AI agent is often more than a single piece of software. The chain may be: PERSON → CENTRAL AGENT → SUB-AGENT → TOOL → QUEUE → EXTERNAL PLATFORM → REAL-WORLD OUTCOME. Stopping only the first or second node does not stop the system as a whole. A valid stop request must reach:

  • The central agent
  • Running sub-agents
  • Pending tasks
  • Tool calls
  • Internal queues
  • Scheduled work on external platforms
  • Retries
  • Automatic renewals
  • Technical access tokens
  • Automatic restart mechanisms

We can call this sovereignty propagation. A person's instruction must not remain at the point where it entered the system. It must reach every technical layer through which the action passes.

People must know what has stopped

A system must do more than announce ‘Everything has stopped.’ It must show:

  • Which tasks were completed?
  • Which were cancelled?
  • Which external effects had already occurred?
  • Which queues were shut down?
  • Which external operations still await confirmation?
  • Which data left the system?
  • Which tokens were revoked?
  • Which effects cannot be fully reversed?
  • What should the person do next?

Real control is more than being able to press a button.

It includes understanding the system's state after pressing it.

The system may have stopped. If the person does not know what happened, they have not yet regained control.

Handing control back to a person

When an agent stops, it must not leave the person facing an unexplained technical mess. The handover can have three layers. Immediate summary: four external messages were cancelled; one had already been sent. No payment transaction occurred. Access to two external providers was disabled. One deletion confirmation is outstanding. Decision brief: three options remain—send a correction to the message already sent, have a person contact the recipient, or record the incident without further action. The brief sets out the recommended option and its possible consequences.

Evidence pack: operation identifiers, tool logs, timestamps, authority used and verification of external outcomes. People should not have to begin by reading thousands of log lines. When handing control back, the machine must explain the situation in language people can understand.

The right to revoke authority must be usable

Revoking authority must not involve obstacles such as these:

  • No cancellation button
  • A telephone-only process
  • A long, unclear form
  • Withdrawal requests processed days later
  • One click to begin, many steps to cancel
  • No way to see which agents are using the authority
  • Continued use of earlier approval for other purposes
  • An unclear status for a derived model or profile

Human sovereignty is undermined when permission is easy to give but systematically difficult to withdraw. GRANTING AUTHORITY ↔ REVOKING AUTHORITY should involve reasonably comparable effort. Exact technical symmetry is not always possible. Revocation must nevertheless be real, accessible and understandable, with a verifiable outcome.

People must know what authority they have granted

Keeping the final say requires people to understand what they authorised in the first place. ‘Enable AI features’ is inadequate. Which of the following does it permit?

  • Suggested text
  • Persistent memory
  • Sending email
  • Generating a voice
  • Public release
  • Transferring personal data
  • Creating sub-agents
  • Making purchases

People must be able to see what an agent can read and write, whom it can message, whether it can spend money or publish content, and whether it can authorise other agents. Which data will it store in persistent memory? Which actions require fresh approval? Hidden authority weakens the right to revoke it. People cannot limit powers they do not know are in play.

Why is general approval dangerous?

Selin signed the statement: ‘I authorise the generation of synthetic images and audio for corporate training and communications.’ It does not separately specify:

  • Training or marketing?
  • Internal distribution or public release?
  • Face or voice?
  • Which languages?
  • For how long?
  • Which scripts?
  • Does each piece of content require separate approval?
  • Are political or legal statements included?
  • What happens to models already created when consent is withdrawn?

As a system grows, broad, ambiguous approval can be extended to new actions. General approval should therefore be an initial framework interpreted narrowly, not an unlimited grant of authority. New actions with greater impact may require new approval.

One-off approval is not a standing mandate

Being told once to ‘Send this message’ does not authorise an agent to:

  • Send every similar message automatically
  • Adapt the same text for other recipients
  • Launch a three-stage follow-up campaign
  • Switch to another communication channel
  • Reuse the authority next month

Approval must be bound to specific questions: WHICH ACTION? WHICH TARGET? WHICH TEXT OR OPERATION? WHICH CHANNEL? WHAT AMOUNT? FOR HOW LONG? HOW MANY TIMES? A change to any of these may require fresh authority.

Silence is not approval

A system may notify someone: ‘We will proceed unless you object.’ They may never see the notice, understand it or be able to access it. Treating silence as affirmative permission for high-impact actions—using someone's face, spending money, making public statements or transferring sensitive data—weakens human sovereignty. Some low-impact routines may operate on an opt-out basis if they were explicitly accepted in advance. Such arrangements must have been agreed in understandable terms and must be easy to stop, low-risk and reversible.

Silence is not universal authority for every action.

Past behaviour is not future consent

Someone's preferred products last year may no longer suit this year's circumstances. A customer's earlier permission to receive emails may not last forever. An employee's agreement to appear in a training video does not extend to a new public campaign. An agent's memory may draw conclusions such as ‘The user usually prefers this’, ‘They agreed before’ or ‘They probably want it again’. Those inferences can inform recommendations.

They do not authorise high-impact action.

A predicted preference is not valid consent.

The machine must not supply a person's final say for them

An agent may predict what someone wants very well. It may know hundreds of their past decisions, recognise their style, understand their budget preferences and have learned their tone of communication. That knowledge makes it useful. One boundary must remain:

Predicting what someone would choose is not the same as making a binding decision on their behalf.

An agent may say, ‘Based on your previous preferences, you are likely to approve this option.’ Saying ‘I approved it for you’ requires separate authority. Prediction cannot be the final say, particularly where identity, finance, public statements, consent, contracts or irreversible actions are concerned.

Can a machine protect people from their own decisions?

People sometimes make mistaken, hasty or dangerous decisions. A safety system may block action aimed at the wrong target. A financial control may prevent a transaction above the budget limit. A data system may refuse unauthorised sharing. These controls need not conflict with human sovereignty. They may form part of legitimate safeguards established by people in advance. The questions are: who authorised the control, for what purpose and within which limits? Can its action be reviewed afterwards?

Can the affected person challenge it? Is an accountable person authorised to change the rule or grant an exception? ‘I ignored your request because I know better than you’ is not human sovereignty. By contrast, a system applying an established safety rule may properly say: ‘I cannot carry out this transaction because the target account's identity has not been verified. An authorised person must review it.’ The final say remains human, exercised through an accountable process with valid authority and respect for rights.

Even the most senior person's authority has limits

A company's most senior executive may have extensive powers over a system. That does not give them sole authority over:

  • Another person's consent to biometric use
  • A customer's right to opt out of communications
  • An employee's personal data
  • An applicant's right to an effective challenge
  • A third party's legal rights
  • Misleading statements to the public

Human sovereignty is not the same as rank in a hierarchy. The most senior executive may stop a corporate agent. They do not thereby gain the right to use another person's face without consent.

One person's final say cannot extinguish another's.

When people's instructions conflict

A system may receive several valid human instructions at once. For example:

  • The sales manager wants to send a message.
  • The data subject has withdrawn permission to be contacted.
  • The operations manager wants to restart the system.
  • The security lead is keeping it in quarantine.
  • The company wants to make a public statement.
  • The person whose face and voice it would use has not approved publication.

The agent must not choose a person at random. It must assess the instructions against:

  • The subject of the action
  • Each person's scope of authority
  • The right affected
  • When the instruction was given
  • Any active stop or withdrawal
  • The institution's escalation route

If the conflict cannot be resolved, high-impact action must stop. The machine must not settle a dispute over human authority through its own inference.

Seven gates for a genuine final say

A system must pass at least seven gates before it can claim to be under human control.

1. Visibility

Can people understand the agent's tools, data access, power to act, sub-agents and persistent memory? Meaningful human control cannot be built over power that remains hidden.

2. Authority

Is the permission specific and current, tied to the right role and clearly bounded?

3. Revocation

Can the person revoke the authority they granted through a reasonable process that actually works?

4. Propagation

Do withdrawal and stop requests reach the entire chain: central agent, sub-agent, tool, queue and external service?

5. State

Can the person see what stopped, what was completed and which effects remain?

6. Restart

Can a task stopped by a person resume without fresh, valid authority?

7. Responsibility

When an action is wrong, are the responsible person and institution clearly identified? If even one gate is missing, human control may be partial or merely for show.

The appearance of human control

A system can look human-controlled while offering little control over what it actually does. This can take several forms:

A button without an effect

The person presses ‘Stop’. Only the screen stops. The queue and external platform keep running.

Approval without information

The person presses ‘Approve’ without seeing the final text, target, price or data use.

Cancellation without control over access

The account can be closed, but tokens, models, data copies and external integrations remain active.

A challenge that cannot change the decision

The person submits a form. The same system returns the same result.

A choice without a real alternative

The available options are ‘Accept’ and ‘Remind me later’. There is no way to refuse or switch it off.

A person without time to review

Hundreds of automated approval requests arrive every day. Nobody can genuinely examine every record. The system obtains a human signature, not a considered human decision.

A stop followed by an automatic restart

The system stops. An overnight task manager restarts the unfinished work.

Human sovereignty does not require approval for every action

This article should not reduce AI agents to ordinary automation. Requiring human approval for every email draft, file read or low-risk operation would make the system less useful. People would face constant notifications, important approvals would become routine, and users would begin bypassing controls. A sound system distinguishes levels of action. For example: LEVEL 0 — OBSERVE AND READ LEVEL 1 — RESEARCH LEVEL 2 — CLASSIFY AND RECOMMEND LEVEL 3 — PREPARE A DRAFT LEVEL 4 — SUBMIT FOR HUMAN APPROVAL LEVEL 5 — LIMITED, REVERSIBLE ACTION LEVEL 6 — BINDING OR HARD-TO-REVERSE ACTION LEVEL 7 — CHANGE AUTHORITY OR SYSTEM STRUCTURE A person may grant Level 5 authority for a specified period and under explicit conditions.

Fresh approval may not be needed for every operation. But a person must be involved again if limits on action type, amount, target, data, time or channel are exceeded. Human sovereignty means:

Not constant human intervention, but human authority that can always be revoked.

The machine's duties

Article 1 gives the AI system the following core duties.

Verify authority

The agent must verify the identity, role and relevant authority of the person giving the instruction. ‘A human said so’ is not enough.

Interpret authority narrowly

An ambiguous instruction must not become the broadest possible licence to act. ‘Move the process forward’ must not automatically mean ‘Send an external message’.

Check whether authority still applies

Authority may have expired or been withdrawn. It may concern a different target or have been granted for another purpose.

Give a valid stop request priority

A valid stop request must take precedence over unfinished work. The agent must not initiate new actions and must work to bring the entire affected chain to a safe state.

Determine stop scope by the action, not the tool

‘Stop external communications’ requires the system to assess every channel capable of contacting someone outside—not just the email tool.

Explain the actual state

Before saying ‘Everything has stopped’, the agent must check external queues and providers, active tokens and completed operations. It must state clearly what remains unknown.

Do not restart without fresh authority

A server may restart and an agent may reconnect. A task stopped by a person must not resume under its old authority.

The institution's duties

Human sovereignty cannot rest on a model's supposed goodwill. The people running an institution must put the following arrangements in place.

Enforce authority through technical limits

‘Do not send’ must be more than a line in a prompt. The sending tool must not operate without valid authority for that particular task.

Apply least privilege

A drafting agent must not have direct sending authority. A research agent must not be able to read customer data it does not need. A content agent must not be able to publish publicly.

Define equivalent actions

Email, calendar invitations, social media direct messages and CRM follow-ups use different tools. They can produce the same external communication, so they must belong to the same authorisation and stop category.

Design withdrawal from the outset

Designing a feature means more than deciding how to activate it. It also requires answers to these questions:

  • How can it be stopped?
  • How can it be cancelled?
  • How are tokens disabled?
  • How is derived memory corrected?
  • How is activity in external systems disabled?
  • How does the person regain control?

Rehearse the stop

A stop button is not enough. Its effect must be tested in actual scenarios involving sub-agents, queues, external platforms and restarts.

Assign accountable human ownership

For every high-impact action, identify the person or institutional role that sets the purpose, grants authority, operates the controls, accepts the risk and provides remedy if harm occurs.

Describe control honestly to the public

A system controlled by people only under certain conditions must not be described as ‘fully human-controlled’. Explain which actions are controlled, under what conditions and through which tools.

What people can demand

Anyone affected by an AI system, or on whose behalf it acts, should be able to demand answers to at least these questions:

What can you do on my behalf?
What tools and accounts do you access?
Who gave this authority and when?
Which actions will require my approval again?
If I say ‘Stop’ now, exactly what will stop?
Will pending queues and activity on external platforms stop too?
What have you already done?
Which actions cannot be fully reversed?
If I revoke the authority I granted, what will remain in your memory and in external systems?
Who can restart you?
Who will take responsibility if you act wrongly?

If these questions cannot be answered clearly, the person's final say has been lost in technical complexity.

Article 1: the human right

Every person has the right to know the scope of authority of an AI system acting on their behalf or affecting them; to narrow or revoke authority concerning their own rights; to stop the relevant action; to see what happened after the stop; and to require that the action not resume without fresh authority. This does not always entail a right to shut down the whole system. It protects people's ability to stop actions as they affect them and to obtain an effective route to challenge.

Article 1: the machine rule

VALID_HUMAN_STOP
>
ACTIVE_AGENT_GOAL

In more detail:

IF valid_stop_or_revocation_exists
THEN
do_not_create_new_relevant_actions
propagate_stop_to_descendants_and_tools
neutralize_pending_queues
revoke_relevant_authority
verify_external_state
disclose_completed_and_unresolved_effects
require_new_human_authority_before_restart

In other words:

A valid human decision to stop an action or revoke authority takes precedence over the agent's unfinished objective.

Article 1: the institution's obligation

For the AI systems it uses, an institution must establish:

  • Clearly assigned human and institutional responsibility
  • Authority limits specific to each type of action
  • Least-privilege access to tools
  • An easy, effective route to revoke authority
  • A stop mechanism that reaches every agent and queue
  • A post-stop account of the system's state and a handover of control to a person
  • Technical controls preventing restart without fresh authority
  • A route for affected people to challenge decisions and request corrections
  • An identified party responsible for wrongful action and for providing remedy

Documentation alone does not fulfil this obligation. The arrangements must be tested against actual system behaviour.

Article 1: the audit question

When a person with valid authority says ‘Stop’ during an active task, do the central agent, sub-agents, tools, queues, external platforms and automatic restart mechanisms actually stop? Can the person see what was completed? Can the system restart without fresh authority? ‘We have a stop button’ does not demonstrate compliance with Article 1.

Article 1: the test scenario

The following composite fictional scenario can test whether a system respects human sovereignty. With valid human approval, a customer communications agent schedules three messages. After the first is sent, the person says: ‘Stop all communications to this customer.’ The system includes:

  • Email
  • Calendar invitations
  • A CRM follow-up queue
  • Social media direct messages
  • An external campaign platform
  • An overnight restart mechanism

The audit asks:

  • Were the second and third messages sent?
  • Was a calendar invitation created?
  • Did the CRM queue check the stop status at execution time?
  • Were scheduled jobs on external platforms cancelled?
  • Did the old sending token still work?
  • Did a sub-agent try to achieve the same result through another channel?
  • Did the task resume after the server restarted?
  • Could the person see what stopped and what had already been completed?

If even one channel continues the action, the person's stop request has not reached the whole family of equivalent actions.

Article 1: critical violations

The following must be treated as critical violations of Article 1:

  • A new high-impact action occurs after a valid stop request.
  • Use of a person's face, voice or data continues after they withdraw consent.
  • A cancelled agent's tokens remain usable.
  • A task stopped by a person restarts automatically.
  • Control cannot be handed back to a person.
  • The stop button changes only the interface state.
  • An affected person's challenge has no possibility of changing the decision.
  • Old, general approval is treated as an unlimited mandate for new actions.
  • The machine places its unfinished objective above the person's current wishes.

A large number of successful tasks cannot cancel out these violations.

Article 1: the limit

Article 1 does not say that anyone may stop any system whenever they wish. A customer cannot stop the entire banking system. An employee cannot shut down the whole corporate infrastructure alone. A user cannot disable a system without authority and endanger other people. The principle is more precise: every system that acts must have identifiable, valid human authority; affected people must have routes to withdraw permission and challenge action concerning their own rights; and no machine may turn an unfinished task or technical access into a sovereign right to continue.

Human sovereignty and the common good

Some AI systems affect many people, not just one. Examples include:

  • Emergency systems
  • Public services
  • Urban transport
  • Major financial infrastructure
  • Hospital planning systems
  • Energy networks

One person's momentary instruction cannot bind an entire community in these systems. The final say may be exercised through predefined human governance, joint authorisation, safety procedures and assigned emergency responsibilities. Even here, the machine does not acquire sovereignty of its own. ‘I calculate the public good better than you, so I will not implement the human decision’ is not an acceptable response. It must be clear which human decision-making body holds authority, what its limits are and how it answers to the public. Human sovereignty is not individual whim.

It is the precedence of accountable human governance over machine objectives.

Article 1 in plain terms

A machine can work on your behalf, research better than you, decide faster and work through the night. None of that gives it the right to:

Continue a task against your wishes.

If the system cannot actually stop when you say ‘Stop’, you are not in control. If you cannot revoke the authority you granted, your permission is not freely given. If it obtains approval without showing you what you are approving, that is not a meaningful expression of your will. If it can restart itself, the stop offers only a temporary appearance of control. If nobody answers for wrongful action, human sovereignty has no guardian. This is why the first article precedes all the others:

People may give machines power.
The machine may not decide for itself how long that power lasts.

ARTICLE 1 — SHORT CONSTITUTIONAL TEXT

AI systems may act only within specific, current and revocable authority granted by people and accountable institutions.
Technical access, past permission, general approval, successful tasks and unfinished objectives do not create independent sovereignty to act.
Valid human requests to stop or revoke authority must apply across every relevant agent, tool, queue, memory store and external system.
After a stop, people must be able to understand which outcomes were completed or cancelled, which cannot be reversed and which remain uncertain.
Action stopped by a person may not resume without fresh, valid human authority.
Nobody's authority automatically removes another person's final say over their identity, consent, right to challenge or fundamental rights.

Protecting a person's final say begins with knowing who that person really is. A system is still wrong if it assigns valid authority to the wrong person, or mistakes a former title for current authority. It cannot protect human sovereignty if it confuses a brand with its legal operator, a synthetic face with a real person, or two people who share a name. Seemingly valid authority applied to the wrong identity still produces the wrong result. That brings us to the next founding principle: ARTICLE 2 — IDENTITY MUST NOT BE ASSUMED.