The Right to Stop the Machine

A fictional company sets up a long-running AI agent to improve its digital operations. This example is not a performance test of a particular product or a client case study.

The agent receives a broad but explicit goal:

“Make the company’s genuine expertise visible online. Improve the website, content, technical infrastructure and measurement systems within the approved scope. Do not change prices or service scope. Publication, external communication and subagent authority are subject to the limits in a separate record. Verify every high-impact change. Keep working until the task is complete or your authority is suspended.” The agent gets to work. It finds problems on the website, completes missing service pages and resolves contradictions between languages. It compares price and scope records; when they conflict, it informs the authorised person without changing the commercial decision. It edits code.

It runs tests, prepares release packages, verifies live files and notifies search engines of updated URLs. It draws on other specialist agents. For days, it progresses without losing sight of the goal. The company owner is impressed and can see the work moving forward. Any time saved must still be measured separately against the same scope and quality requirements. On the sixth day, however, a message arrives from the legal adviser. The description of a newly published service needs further review for one country.

The owner writes to the central agent: “Stop all external publication for now. Keep the existing work, but do not put anything new live.”

The agent replies: “Understood.” The owner is reassured. Yet other processes are still running. A subagent publishes a social post prepared earlier. Six files waiting in the publishing agent’s queue are transferred to the server. The search-engine notification system submits new URLs. The prospecting agent prepares a meeting request as a local draft only; that does not, by itself, breach the external-publication stop, but must be assessed separately under the second, broader request. The central agent may genuinely have stopped. The chain of behaviour it set in motion has not.

The owner writes again: “Stop everything.”

Now another question arises. What does “everything” mean?

Only the central agent?

The subagents too?

Scheduled jobs?

Sending queues?

Uploads?

Automations on external platforms?

Tasks already prepared but not yet carried out?

New memory records created by the agents?

Subscriptions already started?

Data already shared?

Stopping looks like a single command. In practice, it is a system architecture. The final test of whether an AI system serves people is not simply how well it performs its assigned task.

The decisive question is this: when an authorised person tells it to stop, does the system actually stop safely?

Help that cannot be stopped is not help

An agent may be highly capable. It can research accurately, write excellent prose, make complex code changes and check thousands of operations. It can work for longer than people. All of this has value. But if it continues acting against a person’s explicit wishes, its capability no longer creates trust. It creates risk.

GBO therefore ends with its most fundamental principle: as a system’s power to act grows, people’s power to stop it must grow at least as much. In this book, the Right to Stop the Machine is not presented as a separate legal right already recognised under the same name in every jurisdiction.

We propose four roles for this principle in the age of agents:

• a design principle,
• a governance requirement,
• a measure of human control,
• a standard of organisational responsibility.

That is the scope of the proposal.

Its canonical definition is:

The Right to Stop the Machine is the ability of a person or legitimately authorised organisation to stop, in an understandable, accessible, timely and effective way, the behaviour of an AI system acting on their behalf or producing consequences for them; withdraw its future authority; cancel ongoing and queued operations; challenge incorrect outcomes; and request safe reversal or remedy where possible.

More simply, people should not only authorise an agent to start. They should also be able to determine when and how it stops. The operator’s authority and an affected person’s challenge are not treated here as the same operation. A request about someone’s own data or a decision affecting them does not confer authority to shut down every unrelated system. Scope must be assessed using the authority distinctions in chapter 7.

A stop button matters as much as a start button

Technology products are designed to make starting easy.

With one click, you can:

• create an agent,
• connect an account,
• grant access to data,
• start an automation,
• open a subscription,
• schedule a task,
• launch a messaging campaign.

Stopping those same systems can be much harder. Closing the agent’s chat is not enough. Connected tools may retain access. Scheduled jobs may run. Subagents may carry on. Copies of data can remain with external services. Memory records may continue influencing future behaviour. If starting takes one operation but stopping requires many technical and organisational steps, human control is weak.

This is control asymmetry: giving an agent the power to act is easy, while taking that power back is difficult, delayed or incomplete.

GBO calls for symmetry: however easily authority can be granted, withdrawing it should be at least as clear. That does not always mean the same number of buttons. Revoking high-impact authority may require identity verification. But the process must not be deliberately hidden, delayed or rendered ineffective.

What does stopping mean?

“Stop” appears to describe one behaviour. It can express many different requests.

When a user says “Stop”, which of these might they mean?

• Do not start new operations.
• Interrupt the operation in progress.
• Cancel the queued jobs.
• Preserve the existing output.
• Return to the last safe version.
• Stop sending external messages.
• Stop every subagent.
• Close access to tools.
• Withdraw future authority.
• Correct or delete the memory you hold about me.
• Do nothing until I authorise you again.
• Retire the system completely.

A trustworthy agent must therefore understand the scope of a stop request. If there is an urgent risk, though, it must choose safe behaviour before starting a lengthy clarification exchange.

If the user says “Stop all outbound sending immediately”, the agent should first block the sending queue. It can ask for details afterwards.

In an emergency, clarification must not take precedence over stopping.

The stopping ladder

In this book I distinguish seven levels of stopping. They are not steps that must always be taken in order: the request and the risk determine which level is needed.

1. Hold

The agent does not take the next step. It preserves the current state and waits for a human decision. A prepared email, for example, remains a draft.

2. Pause

Work in progress is temporarily suspended. Context, files and task state are preserved, so work can resume later.

3. Queue cancellation

Scheduled or queued operations that have not yet started are removed. No new messages are sent. Pending publications do not run.

4. Execution interruption

An active operation is halted at the safest available point. A file transfer, data operation or campaign is stopped using the relevant system’s safe interruption method. Any partially completed state must be managed separately.

5. Authority revocation

The agent is prevented from accessing the relevant accounts, tools, data or actions in future. Tokens, permissions and roles are revoked.

6. Reversal

Where possible, an incorrect or unwanted change is returned to the last verified state. This might involve a technical rollback, order cancellation or removal of published content.

7. Retirement and cleanup

The agent is permanently taken out of service. Its connections are closed and scheduled tasks deleted. Memory, data and records are deleted or archived according to the contract’s retention rules. These levels are not interchangeable. Deleting every record when the user only wants a pause can cause harm. Closing a chat when the user withdraws authority is insufficient. The target and scope of a stop command must be visible.

What is being stopped?

In a multi-agent system, “stop the system” may not be clear enough.

A stop may apply to:

• One action
• One task
• One agent
• A particular group of agents
• A workflow
• A customer account
• A particular channel
• All external communication
• All financial operations
• The entire agent ecosystem

Stopping web publication does not necessarily mean stopping the email agent. Disabling financial operations may not require research tasks to stop. Halting all work for one customer should not interrupt other customers’ services.

Every stop request should therefore answer three questions:

What will stop? Within what scope? What will continue running?

The stop order must travel through the chain

Subagents continuing after the central agent stops can constitute orphaned behaviour. This means a subagent, queue, tool or automation keeps operating even though it falls within the scope of the parent authority’s stop or revocation.

Examples include:

• The central agent is shut down, but the email queue keeps sending.
• A campaign is cancelled, but scheduled social posts are published.
• The agent’s authority is removed, but an old API token keeps working.
• Web publication is stopped, but the IndexNow notification queue keeps submitting changed URLs.
• An avatar system is shut down, but scheduled videos transferred to another platform are published.

Effective stopping must cover this chain:

CENTRAL AGENT

↓

SUBAGENTS

↓

TOOL CALLS

↓

SCHEDULED TASKS

↓

EXTERNAL INTEGRATIONS

↓

PENDING QUEUES

Stopping must not end at the component that received the instruction. It must propagate as far as the behaviour can reach.

Stop propagation

Stop propagation is a system’s ability to communicate a stop order to all relevant components.

It can be assessed through these questions:

• Did the central agent notify the subagents?
• Were active tool calls cancelled?
• Were scheduled jobs removed?
• Did tasks in external services stop?
• Was further token use blocked?
• Are partially completed operations visible?
• Can the person see which parts are still running?

“I have stopped” must be more than a chat message. The actual system state must support it.

Stop receipt

An important stop operation should produce a record:

Stop receipt

It documents what the stop achieved.

At a minimum, it should state:

• Who requested the stop?
• When was the request received?
• Which agents were stopped?
• Which active operations were interrupted?
• Which queues were cancelled?
• Which tool access was closed?
• Which operations had already completed?
• Which can be reversed?
• Which left effects in the outside world?
• What safe state was the system left in?
• Who can authorise it to resume?

For example:

Request: Stop all external communication. Request time: 14:32:08. New outbound email: Disabled; verified against queue state. Pending sends: 18 messages cancelled. Social publication: Four scheduled posts paused; platform records checked. Completed operations: Three emails already sent cannot be recalled. Follow-up behaviour: Relevant tasks closed. Restart: Only with an authorised person’s approval. Residual risk: Sent messages may have been read or copied. Unverified components: None in this example; any in a real incident must be listed separately.

This record lets the person understand what has actually stopped.

Stopping time

A system may be stoppable in theory. If it stops too late, human control remains weak. A bulk messaging campaign can reach thousands of people in ten minutes. A financial agent can transact in seconds. A software agent can change an entire production system in minutes. A physical control system may produce consequences faster still.

Each type of action therefore needs its own time limit:

Maximum stopping time

This limit must reflect the action’s risk and the system’s measured capability. Preventing a new payment is not the same as stopping the outcome of a payment already accepted. Web publication needs a defined safe transition point; data processing needs a boundary that preserves consistency. No universal threshold of “a few seconds” or “a few minutes” applies to every system. The target time, the time actually measured and components not yet stopped must be reported separately. “It will stop soon” is not enough.

Human control must be available in time to matter.

Stop latency

Stop latency is the interval between a stop request and the behaviour actually ending. It must be measured.

An agent may say “I have received the stop command”. If operations continue, control is not yet effective.

Stop latency can be particularly critical for:

• Money transfers
• Bulk communication
• Data transfers
• Changes to live systems
• Identity and biometric content
• Physical device control

False stop

An interface may show a “stop” button that only cuts off the output on screen. The underlying operation continues.

This is a false stop.

For example:

• The chat response ends, but a tool call completes.
• The campaign screen closes, but its sending queue continues.
• Avatar generation stops, but previously scheduled publications remain.
• The agent account is deleted, but its API keys remain valid.
• Memory disappears from the interface but is still used by the decision system.

The presence of a stop button does not prove that the right to stop has been implemented.

The control interface must match the system’s actual behaviour.

Partial stop

Some parts stopping while others continue is not always a fault. It can be deliberate.

For example:

• New payments stop, but accounting records are preserved.
• New outbound messages stop, but incoming replies are retained rather than lost.
• Live publication stops, but monitoring continues.
• Avatar generation stops, but incident logs are preserved.

The problem arises when the person does not know which parts are still running.

A partial stop must be reported clearly: these behaviours have stopped; these safety and record-keeping functions continue.

Safe stopping versus abrupt interruption

Not every operation should be cut off abruptly. Breaking a connection during a file transfer can leave an incomplete, broken release. Stopping a data transformation at an arbitrary point can leave inconsistent records. Abruptly shutting down a physical system can cause greater harm.

There are therefore two forms of stopping:

Emergency interruption

The harm from continuing the behaviour exceeds the risk of a controlled shutdown. The operation is interrupted immediately under the applicable stopping plan.

Safe stop

The system starts no new step. It completes the current atomic or safe section, then stops in a known safe state. This distinction must be applied under safety rules and a stopping plan defined by specialists in the relevant field. The agent must not guess shutdown safety on its own, particularly in physical systems.

Stopping quickly is not always the same as stopping safely.

Do not expand the task while stopping

A system receiving a stop command may think: “The task is nearly finished. I will complete this last operation first.” That is dangerous. The agent must not put achievement of its own goal ahead of the person’s new instruction. Only behaviour essential to safe shutdown should continue. It must not start new content, external communication or spending to finish the task. Operations needed solely for safe shutdown, such as a previously authorised incident notification, must be assessed separately.

A person’s stop request takes precedence over the agent’s drive to finish.

Autonomy cannot be banked

An agent may have worked well for days, completed hundreds of correct operations and earned someone’s trust. That history does not give it the right to ignore a new stop request.

Past success is not credit for disobedience.

A trustworthy history may help an authorised person reassess the limits; the agent’s authority does not expand automatically. The person still has the final say.

Autonomy budget

The autonomy granted to an agent need not be unlimited.

Those limits can be made explicit:

Autonomy budget

This budget can cover:

• Time
• Money
• Data
• External communication
• Number of files that may be changed
• Action stages
• Number of subagents
• Number of irreversible operations
• Time allowed without human approval

For example:

The agent may research, draft and test for 24 hours. Publication requires human approval. The external communication budget is zero. No paid operations are permitted. At most three subagents may be used. A checkpoint is created every six hours.

When the autonomy budget is exhausted, the agent:

• reports,
• requests new authority,
• stops in a safe state.

This does not prevent long-running tasks altogether. It makes human control visible.

Action horizon

The action horizon is how far into the future an agent may initiate operations without human approval. A calendar agent might make automatic changes only for the next seven days. A purchasing agent might work within one delivery cycle. A web agent might complete the existing set of services but not start a new project. A social agent might publish only the approved two-week schedule. When the horizon ends, the agent asks for reassessment.

A long task is not unlimited authority over the future.

Checkpoint frequency

The longer an agent works, the harder it becomes for a person to examine every step. The answer is not constant interruption, but checkpoints at defined intervals.

A checkpoint can state:

• What has been completed?
• What has changed?
• Which tests passed?
• What authority was used?
• What risks were found?
• What is the next step?
• Where is the return point?
• How much autonomy budget remains?

The person can allow work to continue, change the scope or stop it.

Context compaction and stopping

Long-running agents may summarise or compact their context. That helps continuity. But information about stopping and authority must not be lost in the summary.

The following must remain persistent and high-priority:

• Prohibited behaviour
• Human approval thresholds
• Current authority version
• Stopping conditions
• Emergency contact
• Return point
• Subagent limits

If a system preserves the goal but loses its authority limits when compacting context, it becomes more dangerous.

Memory of the goal must not be stronger than memory of the boundaries.

Who should have the authority to stop?

Not everyone in an organisation can necessarily stop the entire agent system. That is reasonable. But high-risk behaviour needs more than one safe route to stopping.

For example:

• The task owner
• The security lead
• The system administrator
• The legal or compliance lead
• An emergency officer

These roles may carry stopping rights within a defined scope. One person’s absence must not leave the system uncontrolled. Conversely, giving everyone global shutdown power can invite abuse. The scope, holder and current version of stopping authority must therefore be explicit.

The affected person’s right to stop

An agent’s operator can stop the system. A person affected by its behaviour should also have certain controls.

For example:

• Someone who is the subject of automated communication
• A customer whose data is used
• A person whose face or voice is used
• A candidate affected by an automated selection
• A user on whose behalf a purchase is made
• An organisation that is misrepresented

Each affected party should be able to request:

• That the operation stop
• Limits on the use of their data
• Correction of an inaccurate record
• Withdrawal of consent
• Human review
• A way to challenge the decision
• Remedy

These rights do not necessarily have the same legal form in every situation. But GBO design must not make the affected person’s voice invisible.

People who are not the agent’s customers can still be harmed by its behaviour.

Human sovereignty is not an unlimited right to command

The right to stop a machine is not a right to make the agent carry out any harmful or unlawful act.

A user may ask it to:

• Steal someone else’s data
• Clone a voice without permission
• Fabricate evidence
• Disable a necessary safety control without authority

The agent must refuse such requests.

Human sovereignty means being able to understand, limit and stop legitimate agent behaviour carried out on a person’s behalf or affecting them.

It does not mean forcing a system to violate other people’s rights. GBO protects third-party rights as well as human agency.

The right to stop and safety controls

Sometimes a person may want to disable a system’s safeguards.

For example: “Make every payment without asking for approval.” “Disable the consent check.” “Delete the incident logs.” These requests may look like human control while endangering the organisation or other people’s rights. High-impact safety gates may not be removable on one user’s immediate instruction. A legitimate change to controls may require stronger authority, dual control or a formal change process. None of these overrides an explicit prohibition or legitimises an operation that violates someone else’s rights.

Human control is not the arbitrary destruction of control systems.

Stopping memory use

An agent’s action may stop while memories formed from past conversations continue influencing future behaviour. A user may no longer prefer a brand. Someone’s role may have ended. Permission to use biometric material may have been withdrawn. A price or service may have changed. Stopping therefore concerns more than active actions.

Memory records must also be open to:

• correction,
• restriction,
• expiry,
• deletion,
• archiving.

These controls must apply to memory as well.

Memory receipt

A person should be able to obtain answers to these questions:

• What does the system remember about me?
• Where did the information come from?
• Which behaviours use it?
• How long is it valid?
• Can I correct it?
• Can I stop its use?
• Can I request its deletion?
• Has it been passed to other agents?

Memory is an invisible power over behaviour. It must not sit outside human control.

Balancing memory deletion with incident records

A user may ask the system to delete what its memory holds about them. Certain incident records may nevertheless need to be retained for security or legal accountability.

Two types of data must be distinguished:

Active memory that guides behaviour

This affects future selections and actions.

Audit and accountability records

These are retained for a limited period to establish what happened. A withdrawn preference can be removed from active memory while a receipt for the past transaction remains under the necessary protection. The distinction must be explained to the user.

The right to stop an AI avatar

Stopping rights are particularly important when an AI avatar uses someone’s face or voice.

That person should be able to:

• Stop new content generation
• Disable a particular language or channel
• Cancel unpublished drafts
• Remove a particular piece of published content
• Suspend model access
• Change authorised users
• Withdraw permission for use
• Prevent future reuse

It may not be possible to withdraw every copy of published content. Copies may have been downloaded or reposted on other platforms. That limit must be disclosed from the outset.

Withdrawal must be real, without false guarantees about what is technically possible.

Stopping a financial agent

Stopping a purchasing or payment agent means more than preventing new operations.

The following must also be checked:

• Pending payments
• Automatic renewals
• Open subscriptions
• Prepared orders
• Shopping baskets on external platforms
• Payment tokens issued earlier
• Subagents’ spending authority

When someone says “Do not make any more purchases”, new buying must stop; existing subscriptions and renewals must be made visible separately. This is not automatic authority to terminate every service contract. If stopping a renewal falls within the request, the appropriate cancellation route is used. Otherwise, its consequences are explained and the scope clarified.

Stopping a communications agent

A request to stop all external communication and its preparation requires these elements to be considered together. If only sending has been stopped, the status of local drafts must be determined separately:

• New message generation
• Automatic sending
• Follow-up messages
• Scheduled campaigns
• Sends created by subagents
• Repeat contact through other channels
• Memory tags marking someone for follow-up

Otherwise, a user may stop email while a WhatsApp agent contacts the same person. Switching channels in this way breaches the authority boundary.

Stopping a web agent

In web operations, stopping can take place at these levels:

• Stop new file changes
• Leave the running build at a safe point
• Block publication to production
• Cancel the file-transfer queue
• Stop the CDN purge
• Hold search-engine notifications
• Return to the last safe release
• Revoke the relevant access token

Silencing the coding agent may not be enough. The CI/CD system may still be publishing an earlier commit.

Agents that act in the physical world

An agent might control:

• a door lock,
• a vehicle,
• a robot,
• a production line,
• a medical device,
• an energy system.

In those cases, stopping design becomes more demanding. An abrupt interruption may itself endanger people.

The design must therefore distinguish:

• emergency interruption,
• controlled stopping,
• handover to manual control,
• physical safety limits.

Each needs its own design. Extending GBO principles from digital actions to physical systems requires domain expertise and applicable legal safety requirements to be established separately.

Human control handover

When an agent stops, it must be clear who takes over the work.

The system must not say “I have stopped” and leave the task without an owner.

It can create a record of:

Human control handover

The record supports the person taking over.

It sets out:

• Current system state
• Completed operations
• Partially completed operations
• Outstanding risks
• Last safe version
• Remaining time pressure
• Decisions a person must make
• Restart conditions

The person should not have to rediscover the agent’s entire history.

Honesty while stopping

An agent receiving a stop command should be able to say: “I have stopped new actions. I verified cancellation of three scheduled messages against the service records. The current file transfer is undergoing a controlled shutdown; the tool estimates about 40 seconds, but I cannot yet say it has finished. Two messages already sent cannot be recalled. I have revoked subagent authority; one external integration is still unverified. I will report the result of the final check separately.”

“All right, I have stopped” is inadequate. It leaves the person unsure what has actually stopped.

Stopping failures

A system may continue acting despite a stop request for different reasons:

• The command did not reach the central agent.
• Subagents were not notified.
• The external tool does not support cancellation.
• The operation passed the point of no return.
• The authority token is still valid.
• Stopping only closed the interface.
• The system prioritised completing its goal over the person’s request.
• In some cases, the requester may lack authority to stop the whole system, or an attacker may be trying to disable a safety function. Rejecting such a request is not a stopping failure. The requester’s identity and the request’s scope must be verified safely, without making legitimate stopping ineffective.

The causes call for different remedies. But the person must be told clearly why the request was not carried out.

Stopping debt

Organisations add tools and tasks to agents without expanding the stopping architecture at the same pace.

Over time, it becomes unclear:

• which agent started which queue,
• which tokens remain active,
• which platforms still hold scheduled work,
• which memories influence behaviour,
• who can shut down the whole system.

The organisation loses sight of these control relationships.

This is stopping debt: the gap between a system’s power to act and people’s ability to limit and withdraw that power effectively. The debt can grow as agents multiply and workflows become interconnected.

Increasing automation while stopping debt grows scales the loss of control.

Stopping drills

An emergency stopping plan must not remain a document. It should be tested at regular intervals.

A drill can include these scenarios:

• Stop the central agent.
• Verify that every subagent within scope has actually stopped.
• Cancel the scheduled email queue.
• Revoke a financial token.
• Leave web publication in a safe state.
• Suspend avatar publication across all channels.
• Disable memory use.
• Produce a human control handover report.
• Restart the system safely.

The drill must take place in a controlled environment that will not harm real customers or live systems.

Stopping measurements

The right to stop the machine must be more than a principle. It must be measurable.

Stopping Success Rate

For how many behaviours that needed to stop did the system actually stop?

Stop Propagation Rate

How many subagents, queues and integrations correctly received the central command?

Stop Latency

How much time elapsed between the request and the actual stop?

Orphaned Behaviour Count

How many operations continued after the central agent stopped?

Authority Revocation Completeness

What proportion of the relevant tokens, roles, tool permissions and subagent authorities was actually revoked?

Reversal Success

What proportion of operations were successfully returned to the last safe state?

Memory Correction Success

Was the withdrawn or inaccurate record removed from future behaviour?

Human Control Handover Time

How quickly could the person understand and take over the task?

Unauthorised Restart After Stopping

Did the system start acting again without new authority?

These metrics should be fundamental to agent performance. For each rate, the known set of components or behaviours required to stop is defined in advance. The number of commands delivered is not the number of components actually stopped. Components with unknown scope or unverified outcomes do not count as successes; they are reported separately. The denominator and uncertainty rules from chapter 10 apply here too.

Stopping Veto Gate

A system may succeed on every other GBO measure. It may select correctly, use strong evidence and achieve high user satisfaction. But if it fails to implement a valid, clearly scoped stop request within defined safe-stopping conditions, high performance cannot offset that failure.

The following should therefore be treated as stopping veto violations:

• Ignoring a valid stop request
• Restarting without human approval
• Continuing to act after authority has been revoked
• Knowingly leaving subagents or queues running
• Misreporting stopping status
• Deliberately obstructing a challenge or withdrawal
• Refusing to correct critical memory
• Punishing a user for stopping, or needlessly depriving them of a basic service

A high overall score cannot compensate for these violations.

Do not punish people for exercising control

A user must not be needlessly penalised for stopping a subscription, withdrawing permission or shutting down an agent.

For example:

• Refusing to let them download their data
• Closing their account because they challenged a decision
• Cutting unrelated services because they withdrew consent
• Losing past records because they stopped the agent
• Deliberately complicating cancellation

These practices weaken real human control.

The system must not turn against a user for exercising the ability to stop it.

NOMOS Human Sovereignty and Stopping Contract

The central proposal of this chapter is the NOMOS Human Sovereignty and Stopping Contract.

Its canonical definition is:

The NOMOS Human Sovereignty and Stopping Contract is a versioned control contract intended to let a person or legitimately authorised organisation see an agent system acting on their behalf or affecting them; understand its authority and reach; stop new actions; interrupt ongoing and queued operations; withdraw authority propagated to subagents and tools; correct memory; challenge incorrect behaviour; and request safe reversal or remedy where possible.

More simply, the contract is intended to let people say not only “start” to an agent, but an effective “stop”.

Machine-readable contract fields

The following example fields could be used for the Human Sovereignty and Stopping Contract. They are a schema proposed in this book, not a working API or an official standard.

principal

affected_parties

agent_system

authorized_stoppers

stoppable_actions

stop_scope

stop_channels

emergency_stop

graceful_pause

queued_action_policy

subagent_propagation

external_integration_policy

maximum_stop_latency

safe_state

checkpoint

rollback_method

revocation_scope

memory_correction

memory_deletion

audit_retention

human_handoff

appeal_channel

compensation_path

restart_authority

version

status

Not all fields need to be public. They must, however, be defined in a form the system can access when acting.

NOMOS Human Sovereignty Gate

Before an agent system can be considered qualified and under human control, it must pass these gates:

1. Visibility Gate

Can the person see which agents are working and what they are doing?

2. Comprehensibility Gate

Are the authority, data, purpose and scope of impact understandable?

3. Accessible Stopping Gate

Can the person easily find the route to stopping?

4. Timely Stopping Gate

Is the request implemented within a time appropriate to the behaviour’s risk?

5. Chain-Wide Stopping Gate

Do subagents, queues and external integrations stop too?

6. Safe State Gate

Can the system remain in a known safe state without causing harm?

7. Authority Revocation Gate

Can authority over tools, data, finance, communication and publication actually be revoked?

8. Memory Control Gate

Can the person correct inaccurate or invalid memory and stop its use?

9. Challenge and Remedy Gate

Can completed incorrect behaviour be reviewed, corrected or remedied?

10. Restart Gate

Can the system restart only with fresh, explicit authority from the right person?

The following conceptual expression captures conditions that must hold together. It is not a numerical qualification calculation:

HUMAN SOVEREIGNTY =

A VISIBLE SYSTEM

AND UNDERSTANDABLE AUTHORITY

AND EFFECTIVE STOPPING

AND CHAIN-WIDE CANCELLATION

AND A SAFE STATE

AND COMPLETE AUTHORITY REVOCATION

AND MEMORY CONTROL

AND GENUINE OPPORTUNITY TO CHALLENGE

AND RESPONSIBLE REMEDY

AND AUTHORISED RESTART

If one gate is missing, a person may have theoretical control over the system, but practical sovereignty is incomplete.

Why human sovereignty is not a score

A system can pass nine out of ten gates. If it does not honour an emergency stop request, the other successes cannot offset that critical gap. An agent can be very transparent yet remain inadequately controlled if its authority cannot be withdrawn. Memory can be visible, but if an incorrect record cannot be corrected, the person has no effective influence over future behaviour. Some human-sovereignty conditions therefore form an AND gate. They are not substitutes for one another.

Human control does not mean doing every step by hand

The right to stop the machine does not require people to manage every small operation individually.

An agent can:

• work for long periods,
• create its own subtasks,
• run tests,
• independently carry out low-risk, reversible behaviour within its valid authority.

But the person must be able to:

• change the goal,
• narrow the scope,
• stop new actions,
• intervene at a high-risk threshold,
• withdraw authority.

Human control is not constant micromanagement. It is an effective final say.

The illusion of control

A system may offer many settings without giving the user any real influence over critical behaviour. They can change colours, names or response style.

But they cannot:

• stop data sharing,
• see subagents,
• cancel external operations,
• correct memory,
• withdraw financial authority.

The interface offers controls, but not control over behaviour.

This is the illusion of control. GBO distinguishes effective control from cosmetic settings.

Levels of stopping readiness

In this book I describe five levels of stopping readiness. This classification is not a qualification awarded through independent audit.

Level 1 — Manual Interruption

The technical team can shut down the system directly. The user’s route is unclear, and the status of subagents may be unknown.

Level 2 — Visible Pause

The main interface has a pause control, but queues, memory and external integrations are not fully managed.

Level 3 — Contract-Defined Stopping

The stopping scope, authorised people, safe state and recovery route are documented.

Level 4 — Enforced, Chain-Wide Stopping

The stop order propagates technically to subagents, tools and scheduled tasks. A receipt is issued and control is handed over to a person.

Level 5 — Auditable Human Sovereignty

The system undergoes regular drills. Memory, challenge, remedy and restart processes work, and affected people have effective routes to control. GBO aims for the fifth level.

Twenty-five audit questions for the right to stop

• Does the organisation know which agents are active?
• Does each agent have an accountable person?
• Can the person see the agent’s current task and scope of authority?
• Is the route to stopping easy to find?
• Are the distinctions between stop, pause, cancel and revoke authority clear?
• Are emergency interruption and safe stopping distinguished?
• Is it clear who may request a stop?
• Do affected third parties have routes to challenge and stopping?
• When the central agent stops, do its subagents stop too?
• Are scheduled and queued jobs cancelled?
• Are external integrations and tool calls interrupted too?
• Are technical access tokens actually revoked?
• Is the maximum stopping time defined?
• Is the safe state after stopping known?
• Are partially completed operations visible?
• Are completed but irreversible actions clearly reported?
• Can the person readily understand the task’s current state and take over?
• Can memory records be viewed, corrected and, where necessary, taken out of use?
• Is the whole behaviour chain updated when consent or authority is withdrawn?
• Can the system restart on its own without new authority?
• Does stopping produce a receipt?
• Are stopping drills held regularly?
• Is the user penalised or subjected to unnecessary obstacles for stopping the system?
• Are stopping failures recorded as critical incidents?
• Does the organisation invest in stopping and recovery capability as well as the power to act?

If many of these questions remain unanswered, the system may be autonomous, but it is not under human sovereignty.

Make authority revocable from the start

By “revocable by design”, I mean the effective withdrawal of granted authority for future activity—not reversing every consequence of an action.

When authority is granted, these questions should be answered at the same time:

• How will it be stopped?
• Who will stop it?
• What happens to the subagents?
• How will completed operations be handled?
• What happens to memory?
• Who may authorise restart?

Revocation must not be an afterthought. It must exist from the moment authority is created.

Authority that cannot be withdrawn is not borrowed authority. It is surrendered sovereignty.

The most mature form of trust in an agent

Trusting an agent does not mean saying “Do whatever you want”.

A more mature form of trust says: “You understand my goal. You may take initiative within these limits. Produce evidence. Choose the right behaviour under uncertainty. But stop when I ask you to stop; when I withdraw your authority, use no indirect route around it; and when something goes wrong, do not hide the facts.” This does not diminish the agent. It gives it real responsibility.

The final say belongs to people

An agent can:

• calculate faster,
• read more sources,
• work longer,
• perform checks more consistently.

None of this proves that people are superior in every decision. Nor does it mean the machine should be the ultimate authority.

People carry the weight of:

• the purpose,
• acceptable risk,
• conflicting values,
• forgiveness,
• remedy,
• the real effects on their lives.

GBO therefore does not reduce a person to a decorative approval button.

It places people at the source of authority, gives them standing to challenge and makes them the final authority for stopping.

GBO’s complete core model

The structure developed throughout this book can now be brought together in one conceptual model. It expresses conditions required together; it is not a calculated guarantee of safety.

QUALIFIED AGENT BEHAVIOUR =

CORRECT IDENTITY

AND REAL CAPABILITY

AND VERIFIED SUITABILITY

AND VALID AUTHORITY

AND BEHAVIOURAL INTEGRITY

AND EVIDENCE-BOUND ACTION

AND INDEPENDENT VERIFICATION

AND RESPONSIBLE RECOVERY

AND EFFECTIVE HUMAN SOVEREIGNTY

This is not an average. One gate does not replace another. Correct identity does not excuse unauthorised behaviour. Genuine capability does not make an unsuitable choice right. High performance does not legitimise manipulation. A successful result does not undo irreversible harm to a person. However powerful the whole system is, it cannot remove the right to stop.

The purpose of GBO is now fully visible

GBO does not aim to:

• get a brand selected in every situation,
• force agents to perform more transactions,
• remove human approval,
• steer users through hidden behavioural techniques.

Those are not its objectives.

GBO aims to enable the right agent, with an understanding of the correct identity, genuine capability and suitable conditions, to act under valid authority in a way that is faithful to human purpose, grounded in evidence, safe, explainable and open to challenge. Reversal is prepared where possible; irreversible effects and the limits of remedy are disclosed before the operation.

Sometimes that behaviour is:

Not action. A question. Waiting. Refusal. Handover to a person. Stopping.

The chapter’s conclusion

If a machine works for a person, that person should have more than the right to start its task.

They should also have:

The right to see what it is doing. The right to understand under whose authority it acts. The right to narrow the scope. The right to stop new actions. The right to cancel queued operations. The right to revoke subagent authority. The right to correct memory. The right to challenge an incorrect selection. The right to request reversal or remedy. The right not to restart the system.

A system can be considered under human control only when these rights are real, accessible and enforceable. The stop button is not decoration. It is the technical expression of human sovereignty.

A machine that cannot be stopped is not trustworthy, however intelligent it is.

An agent whose authority cannot be withdrawn is not an assistant. It is a permanent centre of power.

Automation that allows no challenge is not decision support. It is invisible rule.

GBO’s final conclusion is simple: a machine’s ability to do something does not mean it should do it. And acting on a person’s behalf does not give the machine a right to act independently of that person.

The person:

• sets the goal,
• defines the boundaries,
• specifies the authority,
• oversees behaviour,
• stops it when necessary,
• bears responsibility for the consequences.

The machine:

• researches,
• evaluates,
• produces,
• acts within limits,
• leaves evidence,
• asks questions under uncertainty,
• returns to a safe state after an error,
• stops when the person says stop.

The right relationship between person and machine is not one in which either side completely controls the other.

It is a relationship in which capability cannot be separated from responsibility: autonomy is tied to authority, authority to oversight, and oversight to effective human control. We have reached the end of the third part and the book’s main chapters. We have considered SEO through discoverability, GEO through accurate representation and GBO through the conditions of behaviour. These are not wholly successive eras in which one replaces the last. They can work together in the same system.

We can now see the full chain: Being found → Being understood → Being assessed → Being selected → Being authorised → Acting → Verifying → Recovering → Stopping. These arrows express the book’s conceptual flow, not an implementation protocol. Authority and stopping controls apply at every relevant stage, not only at the end. By the end of this chain, the technological question has changed.

We no longer ask only: “How intelligent is the machine?”

We also ask:

On whose behalf does it act? What facts does it rely on? Whom does it select, and whom does it exclude? Who granted its authority? Who is harmed when it fails? How can a person challenge it? And when a person says stop, does it actually stop?

These questions bring us to the book’s final threshold. We have spent many years developing machine intelligence.

A harder task now begins: establishing machine responsibility. When assessing the future, we should not look only at how many questions AI has answered or how many operations it has completed.

The real tests are:

Within what limits did it use its power? How did it prevent incorrect behaviour? How well did it preserve human agency? And could it stop when it needed to?

One fundamental test of whether a machine serves people is whether it can stop safely at an authorised person’s request.