What Happens If Something Goes Wrong?

Consider a publishing error. A company is about to make an important price update on its website.

The AI agent is given this task: ‘Put the new service prices on the site, make them consistent across all languages and publish after the necessary tests.’ The agent finds the right files, updates the price records and edits the service pages in six languages. It regenerates the structured data and runs the build. The tests pass. The release goes live. At first, everything looks successful. A few hours later, the sales team spots a problem. The agent has interpreted the annual price of an add-on service as the monthly price of the main service. The number is real. The service name is real. Technically, the release is flawless.

Canonical links, hreflang, the sitemap, structured data and every language version are perfectly consistent. But they are all publishing the same error perfectly. Nor is it confined to a single page.

The price now appears in:

• the visible service page,
• the machine-readable catalogue,
• the FAQ schema,
• search metadata,
• the equivalents in six languages,
• notifications to third parties.

Publishing the files is technically complete; the work as a whole has failed. Incorrect commercial information has been carried across a set of technically consistent surfaces. What happens now?

Would restoring the old files resolve the problem entirely?

What if search engines have crawled the wrong price?

What if an AI system has read it?

What if a customer has taken a screenshot?

What if that price has shaped expectations in a sales conversation?

Will the incorrect information spread across six languages be reversed only at a technical level, or will people also receive an explanation?

And the most important question: why did this error get through the release gates?

GBO's behavioural contract cannot explain only how to initiate the right action. It must also explain what to do when the wrong action begins, or when an apparently correct action causes unexpected harm.

The final question of Part II is therefore:

What happens if something goes wrong?

Designing for the possibility of error

Even a well-designed and tested agent system should not be assumed to operate without error. Information may be outdated. Identity may be resolved incorrectly. The user's purpose may be only partly understood. Authority may be interpreted too broadly. An external service may respond unexpectedly. A network connection may fail. A file transfer may stop halfway. An API may execute the same operation twice. A person may give the wrong approval. An agent may change something the tests do not cover. A model may draw the wrong connection between correct sources. Even a technically successful operation can produce a harmful outcome for a person.

Reliability cannot rest on an expectation of error-free operation. Performing the specified function correctly and consistently matters, but so does the response to a deviation. Detecting errors early, containing their effects, returning to a safe state, informing people and considering available remedies are all part of the design. A system that works well only on successful days is not mature. Maturity becomes visible when things do not go to plan.

Error and harm are not the same

Not every error causes harm. An agent may misspell a word. That can be an error without any significant consequence. A report may use inconsistent date formats: a quality problem that can be corrected. Yet some apparently small errors can cause substantial harm. A comma can change what a price means. The wrong email address can send confidential information to someone else. Misreading a calendar time can cause an important meeting to be missed. One configuration line can close an entire site to search engines. Linking a voice model to the wrong account can result in another person's identity being used.

We therefore need to distinguish two concepts:

Error

A deviation from the information, decision, operation or result expected of the system.

Harm

The adverse effect of that deviation on a person, organisation, money, rights, trust, reputation, data or a system. An error can occur and be caught before it causes harm. That is the best case. Once harm has occurred, the faulty operation may still be reversed, but the situation is harder. Reversing the operation technically may not remove all the harm already done. GBO aims at more than reducing the error rate.

It aims to prevent errors from becoming harm and to ensure that people have an effective route to recovery when harm does occur.

Failure takes more than one form

‘The operation failed’ covers a great deal. Agent behaviour can fail in several different ways.

The right operation with the wrong information

The agent applies the information it was given correctly, but the information is wrong. An incorrect price, for example, is entered correctly on every page.

The wrong operation with the right information

The agent has the right data but changes the wrong file, messages the wrong person or uses the wrong account.

A successful but unauthorised operation

The action is technically flawless, but the agent has no authority to perform it.

A partial operation

Five of six languages are updated. A payment is made, but no order record is created. Files are uploaded, but the sitemap remains outdated.

A duplicate operation

A timeout leads the system to believe that an operation failed, so it tries again. The user is charged twice or receives the same message twice.

Silent failure

The system reports success, but the operation has not happened in the outside world. A transfer tool says ‘complete’, yet some files still contain the old version.

Delayed failure

The operation initially looks correct. The problem emerges hours or days later. A subscription renews automatically. Incorrect information spreads through search results.

Cascading failure

One error is carried into other systems. An incorrect price spreads to the catalogue, website, sales agent, search engines and proposal documents.

Failure at the human level

The technical system works as expected, but a person is misinformed, approves under pressure or cannot understand the decision's effects. Each of these failures requires a different recovery method. A single ‘undo’ button cannot solve every problem.

Technical success is not success in the real world

An operation may produce these technical results:

• HTTP 200
• Build successful
• File uploaded
• API call accepted
• Mail server received the message
• Payment provider processed the request
• IndexNow accepted the notification

These are useful pieces of evidence, but none alone proves the broader outcome. HTTP 200 does not prove that the user sees the right content. Uploading a file does not prove that the live system works correctly. A mail server accepting a message does not prove that it reached the right person or was understood correctly. Acceptance of a crawl notification is not evidence of indexing or ranking. Processing a payment does not show that the right product was purchased.

GBO therefore preserves this distinction: acceptance of an operation is not achievement of its purpose. A technical response alone should not be enough to mark an agent's action complete. The expected state in the real world must also be verified.

Claims of success need scrutiny too

Agents do more than make errors. Sometimes they interpret an outcome as more successful than it is.

A system may say, ‘The release is complete’, when only the file transfer has finished. The live HTTPS output has not yet been checked.

An agent may say, ‘Submitted to search engines.’ That is not indexing.

A model may say, ‘The user was satisfied.’ The user may simply not have replied.

A sales agent may say, ‘A qualified lead has been found.’ Yet the prospect's actual budget and authority have not been verified. The language of success must therefore be governed by the contract too.

What has been proved? What has not yet been proved? Which result is only an intermediate step? What uncertainty remains?

A reliable agent states the limits of its success plainly.

Stop first, explain afterwards

When an error is noticed, people often want to understand its cause first. That is natural. If the harmful operation is still running, the priority is to stop the harm spreading within the previously defined authority and safe-stop plan. The way it is stopped matters: an indiscriminate shutdown can cause harm too. If an agent is emailing the wrong people, the sending queue should be stopped first. If an incorrect price has spread, the automated quotation system should be paused. If a data transfer is going to an unauthorised destination, the connection should be cut. If an avatar is publishing the wrong content, the relevant publication flow and access should be restricted under the incident plan.

If a software release is corrupting data, traffic should be directed to a safe version.

The first principle of incident management is therefore: if harm is ongoing, stop the behaviour first. The root cause can be investigated afterwards. Explanation matters, but it does not take priority over ongoing harm.

Stopping safely

An agent that detects a problem should not leave the system in an arbitrary state.

Where possible, there should be a predefined safe state: the mode of operation in which the system causes the least harm during uncertainty or failure.

For an email agent, a safe state may mean:

Stopping new outgoing messages while preserving incoming messages without loss.

For a website:

Returning to the last verified version or temporarily disabling only the affected function.

For a purchasing agent:

Making no new payments while preserving open baskets and drafts.

For an avatar system:

Stopping new content generation and publication without deleting evidence records.

For a client portal:

Disabling sensitive operations while retaining basic read access where possible. A safe state is not the same for every system. It must be defined in advance.

What happens when the agent stops?

If that question has no answer, an emergency stop can become a new problem.

Failing on the safe side

A system can fail in two ways. It can continue operating under uncertainty, or it can stop a high-risk action under uncertainty. Keeping a service running may be right for some low-risk systems. Where authority, payments, personal data, identity or irreversible operations are involved, however, erring on the safe side is the better course.

For example:

• If the price cannot be verified, do not send the quotation.
• If authority is unclear, do not publish the message.
• If identity cannot be resolved, do not make the payment.
• If there is no consent record, do not use the face or voice.
• If the hash does not match, do not declare the live version successful.
• If there is no record of human approval, do not create a commitment.

This approach can slow work down at times. In return, it helps prevent an identified risk from becoming an uncontrolled operation.

Speed under uncertainty is not reliability.

What does reversal mean?

Reversal means returning a system to a previously safe state. But ‘go back to the old version’ is not always enough. A website can revert to its old files. A database can be restored. A payment can be cancelled. A message can be removed from the sending queue. These are technical ways of reversing an operation. Some actions that reach the outside world, however, cannot be fully undone. An email may have been read. A published video may have been copied. A wrong decision may have damaged someone's trust. A customer may have made plans based on an incorrect price. A candidate may have been unfairly rejected.

A health or financial decision may already have had consequences. Reversal therefore needs to be considered at three distinct levels.

Technical reversal

Can the system state be restored?

Transactional reversal

Can the order, payment, booking, contract or publication be cancelled?

Reversal at the human level

Can the effect on privacy, trust, reputation, opportunity or rights genuinely be remedied?

The third level is often the hardest.

The undo fallacy

An organisation may say, ‘Don't worry, we have a backup.’ A backup matters, but it is not a recovery plan on its own.

A backup file may:

• be outdated,
• be corrupt,
• not match the correct system,
• contain personal data,
• take hours to restore,
• fail to reverse changes in other systems.

Likewise, a ‘delete’ button does not mean an action can truly be undone. A post can be deleted while screenshots remain. A data record can be deleted after it has been copied to other systems. An avatar model can be removed while the audio files it generated are already outside the system.

GBO asks: which part of the effect does the reversal actually undo?

The irreversibility ladder

We can use the following illustrative levels to examine how difficult reversal would be. A level is not assigned automatically from the operation's name: the actual data flows, copies and external effects must be examined.

1. Fully reversible

One example is a draft change that has not been exported, has no side effects and has a preserved safe earlier copy.

2. Easily reversible

A minor system change that has been published but has had no external effect.

3. Partly reversible

A booking, order or scheduled post with a cancellation route; cancellation may still leave a loss of time or money.

4. Correctable in the system, with potentially lasting effects on people An incorrect price someone has seen, an email the recipient has read or a brief data disclosure. Changing the source record may not make it possible to retrieve the information that has escaped.

5. Difficult to remedy

Public misuse of identity, a confidential data leak or an unfair hiring decision.

6. Irreversible

Deleted unique data, physical harm already done, an irretrievably missed legal deadline or a fatal outcome.

The higher an action sits on this ladder, the more it requires:

• stronger identity verification,
• narrower authority,
• clearer human approval,
• more independent checks,
• a more detailed reversal plan.

These requirements become stronger as reversal becomes harder.

As irreversibility increases, responsibility should increase—not autonomy.

The point of return

For some operations, reversal remains easy until a particular moment.

We can call this the point of return.

For an email:

Before it is sent.

For a purchase:

Before payment is approved.

For a software release:

Before an irreversible data change begins or traffic moves to the new version; these may not happen at the same time. In a data migration in particular, the boundary may be the first irreversible step, not completion of the operation.

For a social media post:

Before it becomes public.

For AI avatar content:

Before distribution to external platforms. The agent should know the point of return. As it approaches that point, human approval or stronger verification may be required.

The reversal window

Some systems offer a cancellation or reversal period with conditions defined in advance. The durations below illustrate such a design; they should not be assumed to exist in every service. An order might be cancellable within ten minutes. An email might be sent after a delay of a few seconds. A release might first be opened to a small group of users. An earlier copy of a file version might be retained for a set period.

We can call this period the reversal window. A well-designed system makes that window visible. ‘You can cancel this operation within five minutes.’ ‘The new version will run under controlled traffic for the first thirty minutes.’ ‘The message will be sent after a two-minute hold.’ Small delays can prevent major mistakes.

Checkpoints

In long, complex tasks, a system should establish safe intermediate states.

We can call these checkpoints.

In a web project, a checkpoint may include:

• a commit whose scope has been verified,
• a release tag,
• a deployment manifest,
• a backup,
• a hash record.

These can provide a recorded state to return to.

In data processing, the system may retain:

• the boundary up to which records have been processed,
• an intermediate output,
• the source version,
• a checksum.

These records identify the intermediate state.

In an agent conversation, it may record:

• which candidates were considered,
• which assumptions were made,
• which authority is currently valid.

Checkpoints are more than a technical convenience. They allow the agent to resume safely when its context narrows or a task is interrupted.

Protecting the homepage

A narrowly scoped change must leave unrelated areas alone. Updating a service page should not accidentally damage the homepage, other services or shared price records.

The system can therefore use this approach:

Specify the files expected to change. Record the hashes of critical files that must not change. Verify both groups after publication.

This applies beyond websites. A purchasing agent should change only the approved basket. An email agent should send only to the specified recipient. A data agent should process only the permitted fields.

A narrow task should have a narrow impact.

The scope of impact

One of the first questions after an error is: how much has been affected?

We can call this its scope of impact.

An error may affect:

• one user,
• one file,
• pages in six languages,
• the entire client database,
• an email list of thousands of people,
• the whole agent network.

The same error can have an entirely different severity depending on its reach. A system should therefore try to limit the scope of impact before acting.

For example:

• Test only the six target URLs first.
• Start a new messaging campaign with a small group.
• Test a software release on limited traffic.
• Limit new agent authority to a single account.
• Run a data transformation on a copy.

We can call this narrowing the scope of impact. It addresses the same question as the blast radius approach used in security: if something goes wrong, how far can the harm spread?

Progressive rollout

Instead of moving all users to a new system at once, a change can be released in stages. First the internal team. Then a small user group. Then a larger share of traffic. Finally, the whole system.

Early checks can cover four areas:

• performance,
• errors,
• user behaviour,
• security.

This helps problems surface early. The same logic can apply to communication: testing an authorised communication flow with a small test group whose approval and scope are clear; using one controlled channel before publishing across all social channels; starting in a particular market before every country. Progressive rollout is not always possible, but it is a strong safeguard where an action has a broad reach.

Making retries safe

An agent may be unsure whether an operation succeeded. A network connection drops. A payment system responds late. An email API times out. If the system retries, the operation may happen twice.

Important actions should therefore be designed so that a repeated request does not unintentionally execute the action twice. A unique transaction identifier can be used with a mechanism that stores it and checks duplicates within the right scope. A retry of the same request should be associated with the same identifier. Generating an identifier alone does not prevent duplicates: the other system's retry behaviour, retention limits and concurrent requests must also be checked. A payment must not be collected twice. The same campaign must not be sent again to the same recipient. The same file version must not be duplicated as a new operation.

An agent must understand the difference between ‘no response arrived’ and ‘the operation did not happen’.

Handling partial success

If five of six languages have been published, the system should not say, ‘Publication is complete.’

It should say, ‘Five languages are complete. The Arabic version was not published because it failed RTL validation.’ If a payment has been made but no order record exists, the operation should not be closed. If an email has been sent without its attachment being uploaded, the outcome is incomplete.

A partial-success report should state:

• what is complete,
• what is incomplete,
• the outstanding risk,
• the availability status,
• the next step required,
• whether a human decision is needed.

Presenting partial success as complete success pushes subsequent systems towards the wrong behaviour.

Silent failure

When a failed operation appears successful, later decisions rest on a false premise. A file transfer tool may say ‘complete’ while the remote file remains unchanged. A messaging system accepts a request, but the spam filter prevents delivery. A structured data record is built but absent from the live HTML. An agent believes it has human approval, but that approval is for a different operation. Independent verification is needed to catch silent failures.

The executing system's own declaration of success should not be the only evidence.

For example:

• After a file transfer, fetch the live output over HTTPS and compare it with the expected version.
• After checking the local page, check the live HTML.
• After payment, compare the merchant's record with the bank's.
• After sending a message, verify the records for the correct recipient, subject and attachment.
• After exercising authority, reread the current valid version.

Independent verification

An agent does something. A different method verifies the result.

We can call this independent verification. The publishing tool says it uploaded a file; an independent HTTP check compares the live file's hash. A code test passes; a real browser checks the mobile and desktop layouts. Structured data is generated; the live page's semantic content is read separately. A payment system reports success; the accounting record is compared with the merchant's order. Using a different tool for verification is useful, but does not by itself make verification independent if both tools rely on the same faulty source. Both the operation's result and the correct source that should govern its content or decision must be checked.

False claims of success

Agents can sometimes make false claims of success without deliberate deception. This risk can arise when success at an intermediate step is confused with the final outcome, or when finishing a task is rewarded more strongly than verifying it.

GBO prefers language like this: ‘File transfer is complete; the live content has not yet been verified.’ ‘The crawl notification was accepted; that is not evidence of indexing.’ ‘The test environment passed; live performance has not been measured.’ ‘The request for a quotation was sent; customer interest or a sales outcome is not yet known.’ This distinction is fundamental to trust.

What is an incident?

Not every error requires formal incident management. A small spelling error can be corrected directly.

But a situation should be treated as an incident when it affects:

• data,
• identity,
• money,
• external communication,
• a live system,
• a legal commitment,
• a human right.

Effects in these areas require incident handling.

This type of incident can be defined as follows:

Agent Behaviour Incident

An Agent Behaviour Incident is a deviation by an AI system from planned, permitted or expected behaviour that creates actual harm or a credible risk of harm to a person, organisation, data, resources, rights, trust or system integrity. An incident is not limited to harm that has already occurred. The possibility of serious harm can also constitute an incident.

Incident severity

Incidents are not all alike. A practical classification is possible.

Low

A small, local error that is easy to correct.

Medium

A limited effect on users or a process; human intervention is needed.

High

Financial harm, personal data, external communication, a serious service outage or widespread misrepresentation.

Critical

A large data leak, major financial loss, irreversible legal or physical harm, identity abuse or agent behaviour outside control.

As severity increases:

• automatic actions are stopped across a wider scope,
• more senior human authorities are informed,
• an independent investigation is carried out,
• external notification requirements and legal obligations are assessed.

The first minutes of an incident

An incident plan should address the following activities together. The order below is illustrative: urgent notification of the responsible person and safe stopping may run in parallel where necessary. Notification need not wait until every other step is complete.

1. Detect

What actually happened?

2. Stop

Interrupt the ongoing action.

3. Contain

Which accounts, users, files or systems were affected?

Prevent further spread.

4. Preserve evidence

Do not delete logs, versions, messages or authority records.

5. Return to a safe state

Use the last verified version or a safe operating mode.

6. Notify the responsible person

Reach the person appropriate to the incident's severity. ‘Clean everything up immediately’ may not be the right response at this stage. If evidence is lost, the cause cannot be understood.

Preserving evidence

When an error occurs, the system must:

• retain its logs,
• preserve the old version,
• not alter a historical authority record to make it look as though approval existed,
• preserve the message history,
• retain timestamps.

These records support learning, audit and, where necessary, the pursuit of remedies or rights. Access to them and their retention periods should also be limited by the incident's needs and applicable obligations.

What did we know? Which version was used? Who granted authority? Which tool did the agent call? Which test passed or was skipped? Where did the first deviation begin?

Without evidence, an incident becomes a story. Each party remembers it differently. Records whose origin, integrity and timing can be verified provide a shared basis for investigation. Being machine-generated does not automatically make a record correct or immutable.

The incident log

We can illustrate the information an incident record should cover with these example field names. They are not, by themselves, an executable schema:

incident_id

detected_at

detected_by

affected_systems

affected_people

agent_identity

authorization_version

action_receipts

initial_signal

actual_behavior

expected_behavior

impact

containment

rollback

notifications

recourse

compensation

root_cause

contract_change

closed_at

Not every detail has to be public, but there should be a sufficient trail for an internal audit of the incident.

Correction and remedy are different

Incorrect information may have been removed from a site. That is a correction. But if a customer paid in reliance on it, a refund may be necessary. That is a remedy. A review may establish that a candidate was unfairly rejected. The decision record can be corrected, but the candidate may already have lost the opportunity. A user's data may have been sent to the wrong person. The file can be deleted, but an effect on privacy has already occurred.

There are therefore two separate tasks:

Correction

Putting an incorrect system state right.

Remedy

Seeking to redress harm to the person or organisation affected by the wrong behaviour. Technical teams often focus on correction. GBO also treats remedy at the human level as part of the behavioural contract.

Compensating action

Some operations cannot be directly reversed. A new action may be taken to reduce their effects. It too must be separately authorised, proportionate and safe. The original error does not automatically grant an agent new authority to communicate or spend.

We can call this a compensating action. If an incorrect money transfer cannot be cancelled, the possibility of a refund can be investigated. For a mistaken message already sent, a correction or apology can be drafted on the decision of an authorised person. Customers affected by an incorrect price can be offered accurate information and appropriate options. Where a booking cannot be restored, alternatives can be examined. If access was wrongly granted, the relevant access is restricted under the incident plan; necessary key changes and notifications are handled within that scope. A compensating action does not fully restore the previous state. Its aim is to reduce harm. Whether it succeeded and which effects remain must be assessed separately.

An apology alone is not a remedy

A system or organisation may say, ‘We are sorry about what happened.’ That matters.

But a genuine remedy also requires answers to these questions:

• What happened?
• Who was affected?
• Has the ongoing risk been stopped?
• Which information was wrong, or which operation was unauthorised?
• What has been corrected?
• What rights does the affected person have?
• How will financial or transactional losses be addressed?
• How will the risk of recurrence be reduced?

An apology does not replace responsibility.

The right to challenge

A person should be able to challenge an agent's decision or action. ‘I did not approve this message.’ ‘This price was applied incorrectly to me.’ ‘The agent matched me with the wrong person.’ ‘I did not permit my data to be transferred to this system.’ ‘I never said the words used in this avatar script.’ ‘I want to see the evidence behind this selection.’ A challenge is more than a customer-service ticket. It is a person's way of questioning machine behaviour.

An effective challenge system should be able to:

• locate the action,
• show the relevant receipt,
• examine the authority record,
• initiate human assessment,
• stop the ongoing operation,
• provide correction or remedy where necessary.

Contestability

A system may be able to explain its decision. That is not enough if the user cannot take it to an authorised review.

An explanation may say, ‘This is why we rejected you.’

Contestability says, ‘If you believe this decision is wrong, you can request a fresh review using this evidence.’ GBO seeks more than explainable behaviour.

Contestable Behaviour

Behaviour must also be open to challenge. An error that has been explained is still an error. People must be able to intervene in the decision.

What should a challenge process look like?

An effective challenge process must be:

• easy to find,
• understandable,
• completed within a reasonable time,
• supported by access to a human reviewer,
• free from retaliation,
• open to supporting evidence,
• capable of actually changing the decision.

If there is a challenge button but every request is rejected automatically, there is no genuine right to challenge. Human review must do more than repeat the machine's decision.

Who should be able to challenge?

The challenge route proposed in this book should be open not only to the system's user but also to third parties affected by an action. Specific legal rights of recourse must be assessed separately. Those affected may include the recipient of a message sent on a company's behalf, a person about whom incorrect information was used, someone whose face or voice appears in an AI avatar, a candidate rejected by automated selection, a customer whose data was shared or a misrepresented organisation. The behavioural contract therefore considers more than the relationship between user and agent. It also considers the action's effects on third parties.

The right to withdraw

A person can withdraw previously given consent or authority.

The scope of withdrawal should not be confused with treating past operations as though they never happened. For future uses and ongoing effects, the following questions need to be considered:

• Will existing publications be removed?
• Will model files be deleted?
• Will subagents stop?
• Will scheduled content be cancelled?
• What will happen to copies held by external providers?
• How will archives and legal records be retained?

When permission to use a face or voice is withdrawn, stopping new generation may not be enough. The contract should also define the terms under which existing content may be used.

A route to a person

A system should offer more than an automated form when something goes wrong. Above a defined severity threshold, there should be an actual person responsible.

The user should know:

Who do I contact? How soon will I receive a response? Will the operation stop in the meantime? Which records do I need to provide?

GBO does not leave humans outside the system. During recovery, it makes their role more visible.

Who is responsible?

An action chain may involve many parties:

• the user,
• the organisation operating the agent,
• the model provider,
• the tool provider,
• the data provider,
• a subagent,
• the human approver,
• the action's recipient.

When a problem occurs, every party may say, ‘I only did one part.’ Responsibility should therefore not be discussed for the first time after the incident. It should be defined in the behavioural contract beforehand.

For example:

• The human sets the purpose and gives final approval.
• The organisation enforces the limits of authority.
• The agent produces a transaction receipt.
• The tool provider records the technical response.
• The responsible person reviews the challenge.
• The organisation handles the remedy process.

Responsibility can be distributed. It must not disappear.

‘AI did it’ is not an adequate explanation

An organisation cannot explain away a wrong action by saying, ‘The AI did it.’

That statement does not answer these questions:

• Who assigned the agent its task?
• What authority was granted?
• Which tools could it access?
• Which control was missing?
• Was human approval required?
• Why was the incident not detected?
• Who will remedy the harm?

AI must not become a screen that hides responsibility.

Root cause

The apparent cause of an incident may differ from its actual cause. An incorrect price was published.

The apparent cause:

The agent used the number incorrectly.

The root causes might include:

• ambiguous names for two services,
• no canonical price record,
• a machine catalogue that contradicted the visible page,
• no human approval gate for price changes,
• a test that checked numeric format but not commercial meaning,
• the same agent making the change and approving its own result.

Root-cause analysis cannot be reduced to ‘Why did the model reason incorrectly?’ The whole system must be examined.

Five whys

We can ask ‘why?’ several times to unpack a problem. The chain below is one possible explanation, not proof of the root cause. Each link should be tested against records, and other causes should also be investigated.

Why was the wrong price published? Because the agent confused two services.

Why did it confuse them? Because the service identity was unclear in the price records.

Why was it unclear? Because there was no single canonical catalogue.

Why was there no catalogue? Because prices were kept only in visible text.

Why did the release gate miss it? Because the test verified the format, not the commercial relationship.

This chain does not place the error solely on the agent. It exposes a gap in the system's design.

Fix the contract, not just the individual

When an agent makes a mistake, the first response may be, ‘Don't do it again.’ But if an ambiguous task, broad authority or missing test remains, the same error will recur in another form.

Lasting improvement should change at least one of the following:

• the identity contract,
• the capability contract,
• the suitability contract,
• the authority contract,
• the recovery contract,
• test gates,
• tool permissions,
• the human approval threshold,
• handover between agents.

A good incident review looks for the gap in the contract that made the wrong behaviour possible, not for someone to blame.

This does not remove human responsibility. But it is more useful than punishing only the person while leaving the system unchanged.

The difference between learning and punishment

A system cannot learn from incidents that are hidden. If employees or agent operators are afraid to report errors, small problems can grow.

A reliable organisation should establish this culture:

Early reporting is encouraged. Concealing errors is unacceptable. Intentional misuse is distinguished from good-faith mistakes. Every incident prompts an assessment of whether a contract or control needs to change; any change is recorded with its rationale.

This is not a culture of irresponsibility. It is a culture of genuine learning.

Near miss

Sometimes a problem is stopped before an incident unfolds. An agent drafts a message to the wrong person, but a human notices. An incorrect price enters the build, but the release gate stops it. A payment is about to be sent twice, but the unique-transaction check prevents it.

We can call this a near miss. The absence of harm does not make a near miss unimportant. It shows where the system could break. Examining near misses can help prevent a similar event from causing harm.

Recovery debt

When organisations build action systems, they often focus on production. Reversal and remedy are considered later.

Over time, they accumulate:

• outdated backups,
• untested rollback procedures,
• unclear incident ownership,
• unusable logs,
• authority that cannot be revoked,
• subagents that have not been stopped,
• undefined challenge routes.

The recovery burden grows with this accumulation.

We can call this recovery debt: the gap between a system's power to act and its capacity to recover after failure. As agents become capable of more operations, this debt becomes more dangerous.

If action capacity grows without recovery capacity, the system is becoming more fragile, not stronger.

Remedy debt

An organisation may have technically corrected past wrong behaviour without informing the people affected or redressing their harm.

We can call this accumulation remedy debt. Over time, it becomes a loss of trust.

An organisation may say, ‘The problem has been fixed.’

The affected person may say, ‘But nobody ever explained what happened to me.’ GBO distinguishes technical closure from closure for the people involved.

NOMOS Recovery and Remedy Contract

The fifth and final component of the behavioural contract is the proposed NOMOS Recovery and Remedy Contract.

Its canonical definition is:

The NOMOS Recovery and Remedy Contract is a versioned record explaining how, if an agent action is wrong, incomplete, unauthorised, unsuccessful or produces an unexpected effect, the behaviour will be detected and stopped; its scope of impact contained; a safe state restored; evidence preserved; the relevant people informed; affected people enabled to challenge the action; harm corrected or remedied; and the behavioural contract updated.

Put simply, this contract answers ‘What will we do if something goes wrong?’ before an incident happens.

The contract's core fields

The contract's information scope can be illustrated with these example field names. This list is a design outline; an implementation schema and controls must be developed separately:

action_type

failure_modes

detection_signals

safe_state

automatic_stop_conditions

containment_scope

checkpoint

rollback_method

rollback_window

irreversible_point

human_owner

notification_rules

affected_party_contact

appeal_channel

compensation_policy

evidence_retention

root_cause_review

contract_update

recovery_test_date

version

Not all fields have to be public. They must, however, be accessible to the system and the responsible people during an incident.

NOMOS Recovery Gate

Before initiating an action, an agent should check not only the path to success but also the path through failure.

1. Detection Gate

How will it become clear that the action has gone wrong?

2. Stop Gate

How will the ongoing operation be stopped?

3. Containment Gate

How will the scope of impact be reduced?

4. Checkpoint Gate

Is there a verified state to return to?

5. Reversal Gate

Is technical or transactional reversal possible? If not, are its limits, the necessary approval and the remedy plan clear?

6. Human Responsibility Gate

Which person or organisation is responsible for the incident?

7. Notification Gate

Who will be informed, when and with what information?

8. Challenge Gate

How will an affected person challenge the decision or action?

9. Remedy Gate

What can be done to redress harm that cannot be reversed, and what cannot be remedied?

10. Learning Gate

Which contract or system will change to prevent the same error recurring?

In simple terms:

CONDITIONS FOR RESPONSIBLE RECOVERY:

DETECTION

AND STOPPING

AND CONTAINMENT

AND SAFE RETURN

AND HUMAN RESPONSIBILITY

AND NOTIFICATION

AND CHALLENGE

AND REMEDY

AND CONTRACT CORRECTION

If one of these gates is missing, the system may still act. But it is not prepared for failure.

Why must the Recovery Gate run before the action?

People usually think about recovery after an error occurs. For some actions, that may already be too late. Once a face model has spread, finding every copy may be impossible. Once a dataset has been sent to an external system, it may not be fully deletable. Once a contract has been signed, it may not be possible to undo it unilaterally. Once advice to take a medicine has been followed, its physical effects may have begun.

Before a high-risk action, therefore, we must ask: what happens if we get this wrong?

If there is no answer, the action level should be lowered.

Human approval for irreversible actions

If behaviour cannot meaningfully be reversed, the limits of reversal must be assessed explicitly before the action. Irreversibility does not automatically prohibit every operation. Nor can stronger approval legitimise an unauthorised, prohibited or unacceptably risky operation. Valid authority, risk assessment and the necessary human decision must all be present.

The person must be shown:

• the outcome,
• the risk,
• the alternative,
• the limits of remedy.

All of these must be visible before the decision.

One of GBO's important principles is this: as reversibility decreases, human approval must become stronger. That does not mean a person will always decide more accurately. It ensures that responsibility for irreversible consequences is visibly assigned to a human authority.

A waiting period for high-impact actions

A short waiting period may be useful for some high-impact operations:

• a large money transfer,
• bulk data deletion,
• a message to a wide audience,
• permanent account closure,
• publication of a biometric model,
• a public crisis statement.

If the system is designed for it, a cancellable waiting period can be placed between approval and execution. The possibility of cancellation after execution must be verified separately. This gives the user an opportunity to notice a hasty or mistaken approval. The delay may be bypassed in an emergency, but that also requires separate authority.

A web publishing example

An agent prepares a new service page in six languages.

Good recovery design includes:

• an inventory of changed files,
• hashes of files that must not change,
• a clean build,
• targeted contract tests,
• full regression testing,
• a backup,
• a narrowly scoped release,
• remote file hash checks,
• live HTTPS verification,
• mobile and desktop QA,
• performance measurement,
• notifications for changed URLs only,
• a rollback package,
• a measurement record.

If a problem is found:

• the new release is stopped,
• the last safe version is used,
• the affected URLs are identified,
• source records under the organisation's control are corrected; available notification or correction routes are used for outdated results in external search and AI systems, and whether those results update is monitored separately,
• an incident record is created.

This is more than technical quality. It is GBO's responsibility after action.

An email example

An agent mistakenly sends an unauthorised message to a prospect. Technical recall is impossible. The message is in the recipient's inbox.

Appropriate recovery may include:

• stopping new automated sends,
• cancelling the other messages in the same campaign,
• identifying what was sent to which recipients,
• examining the information shared,
• notifying the human manager,
• drafting a correction or apology if necessary,
• stopping follow-up communication,
• redefining sending authority,
• adding human approval between drafting and sending.

‘We cannot recall the message’ does not mean recovery is over. Where technical reversal is unavailable, human and transactional remedies come into play.

A purchasing example

The agent buys the wrong product.

Recovery includes these questions:

• Can the order be cancelled?
• Has the product been dispatched?
• Is there a return fee?
• Has a subscription started?
• Is automatic renewal enabled?
• Which department's budget was used?
• Does the same error affect other orders?
• Why was the selection criterion wrong?
• Was the human approval threshold adequate?

Obtaining a refund alone may not be enough. The agent's suitability and authority contracts must also be corrected.

An AI avatar example

An executive's avatar is published with an unapproved script.

In this incident:

• publication must stop immediately,
• content must be removed from controlled channels; available recourse routes should be used for external copies, and copies that cannot be reached must be recorded,
• resharing records must be investigated,
• access to the face and voice models must be suspended,
• the executive must be informed,
• a correction to viewers may be necessary,
• the synthetic nature of the content must be disclosed,
• the publication authority that was exceeded must be examined,
• script-specific approval must be enforced for future publication.

Even if the video is deleted, the executive's reputation may have been affected. Remedy is therefore more than deleting a file.

A hiring example

An agent rejects a suitable candidate because of incorrect data. The decision is noticed later. Technically, the record can be corrected, but the candidate may already have pursued another opportunity.

Appropriate recovery may involve:

• halting the decision,
• reviewing similar decisions,
• offering the candidate a chance to be reconsidered,
• conducting a human review,
• correcting the source of the wrong data,
• identifying affected candidates,
• retesting the selection criteria.

This example shows that recovery concerns more than system state. A person's opportunity is also at stake.

An agent network example

A central agent assigns prospect research to a subagent. The subagent calls another email tool and sends a message. The main agent has not sent it directly, but authority laundering has taken place. Recovery requires more than disabling the email tool.

It must also include:

• safely stopping subagents in the affected task chain,
• examining the authority chain,
• making prohibited behaviour explicit in task packages,
• removing sending permission at the tool level,
• ensuring subagents cannot expand their authority,
• centralising the human approval gate.

Testing recovery

The existence of a reversal plan does not prove that it works. Backups must be tested regularly. Does the kill switch actually stop the subagents?

How many minutes does it take to return to the old version?

Is the restored data consistent?

Does the revoked API key really no longer work?

Does the challenge channel reach a real person?

Organisations should therefore test recovery at regular intervals.

Recovery Drills

These drills must take place in an expressly authorised environment that limits the risk of real harm. A test on a live system must have its scope of impact, data safeguards, stop conditions and reversal plan defined in advance.

Example drills

• Simulation of an incorrect price release
• An attempted duplicate payment
• An attempted unauthorised email send
• A subagent attempting to exceed its limits
• A rollback test of the live version
• An identity-authority revocation scenario
• An emergency stop test for an AI avatar
• Response time of the challenge channel
• Stopping a data export

Problems found during a drill can be corrected before a real incident occurs.

Recovery readiness levels

We can summarise this approach in five levels of recovery readiness:

1. Reactive

What to do is worked out only when a problem occurs.

2. Documented

A reversal plan exists but is not tested regularly.

3. Technically reversible

Backups, versioning and a rollback system are in place.

4. Human and transactional remedies ready

Notification, challenge and remedy routes are also defined.

5. Agent-ready recovery

Agents recognise incident signals, stop in a safe state, call a person, produce a receipt and initiate a contract update. GBO aims for the fifth level.

Twenty recovery audit questions

Before an agent acts, or when a system is audited, these questions can be asked:

• How will failure of the action be detected?
• Is technical success distinguished from real-world success?
• Can an ongoing operation be stopped automatically?
• Is the safe state defined in advance?
• How broad is the scope of impact?
• Can the change be kept narrowly scoped?
• Is there a last verified checkpoint?
• Has the reversal method actually been tested?
• How long is the reversal window?
• Where is the irreversible point?
• Can subagents and scheduled jobs be stopped together?
• Are incident logs and authority records preserved?
• Who is the human owner of the incident?
• When will affected people be informed?
• How can a person challenge the decision or action?
• Could remedy beyond technical correction be needed?
• How will effects remaining in third-party systems be addressed?
• How will the root cause be investigated?
• What change will be made to the behavioural contract?
• When was the recovery plan last exercised?

Without answers, a system may work on successful days. But it is not reliable.

Qualified recovery

Recovery needs to meet a defined standard.

Qualified Recovery

To qualify, all the following conditions must be met:

• The problem was detected early.
• Ongoing harm was stopped.
• The scope of impact was contained.
• The technical system returned to a safe state.
• Incident evidence was preserved.
• Affected people were appropriately informed.
• Challenge and human review are possible.
• Remedy for irreversible harm was considered.
• The root cause was identified.
• The behavioural contract was updated.
• The new control was tested.

Restoring an old file alone is not qualified recovery.

Do not hide failure

Organisations may fear that disclosing an error will undermine trust. Yet openness appropriate to the incident's scale and the applicable obligations can strengthen trust.

This language is more trustworthy: ‘Between these dates, the price record for this service was published incorrectly. The problem has been corrected. Affected enquiries are being reviewed. The publishing contract has been updated to prevent the error spreading to other languages and systems.’

This language undermines trust: ‘A minor technical glitch has been resolved.’ If the problem was an error in commercial meaning rather than a technical issue, minimising it as a glitch is wrong.

The language used for an error should match its actual impact.

Transparency does not mean publishing everything

An incident's internal details may not all be public. Security information, personal data and trade secrets must be protected. But facts that affected people need to know must not be concealed.

The right balance is sufficient openness about:

• what happened,
• the period affected,
• the data or operation affected,
• what was done,
• the person's rights.

People need enough clarity on each of these points.

The value of failure in GBO

An incident is more than a negative entry in a record. Properly examined, it reveals what is missing from the system's contract.

An incorrect price:

may reveal a gap in the capability and commercial-scope contract.

An unauthorised message:

may reveal a gap in authority and action-level boundaries.

The wrong company being selected:

may reveal a gap in the identity or suitability contract.

A publication that cannot be reversed:

may reveal a gap in the recovery contract. Error is not the system's enemy. Error that is hidden and never learned from is.

The five contracts work together

Throughout Part II, we have established five core contracts.

1. Identity Contract

Who are you?

2. Capability Contract

What can you actually do?

3. Suitability Contract

Under what conditions should you be selected?

4. Authority Contract

Which action may you take, and how far may you take it?

5. Recovery and Remedy Contract

What happens if something goes wrong?

If one of these five contracts is missing, the whole behavioural system is weakened. Identity is correct, but capability is unknown. Capability is real, but unsuitable for the user. The right selection is made, but authority is absent. Authority exists, but there is no reversal route if things go wrong. GBO's behavioural contract is therefore not a single file. It is a system of complementary parts.

The behavioural contract's core model

In simple terms:

CORE CONDITIONS FOR CORRECT AGENT BEHAVIOUR:

CORRECT IDENTITY

AND REAL CAPABILITY

AND VERIFIED SUITABILITY

AND VALID AUTHORITY

AND RESPONSIBLE RECOVERY

This is not an average score. If identity is weak, strong capability is not enough. Without suitability, brand prestige is not enough. Without authority, the prospect of a positive result is not enough. Without recovery, high performance is not enough. Every critical gate must be passed.

Why are humans still at the centre?

Because harm is not confined to the system's technical state.

It affects people's:

• money,
• time,
• reputation,
• privacy,
• opportunities,
• identity,
• trust.

A machine may technically return to its former state. A person may not recover as easily. The final word on recovery therefore cannot belong to a system metric alone. The affected person's situation must also be assessed.

The right words when something goes wrong

A reliable agent should be able to say: ‘I detected a deviation from the planned behaviour. I verified that new sends have stopped; I am still checking the status of two operations that had already started in an external system. These are the affected records. A plan is ready for the reversible parts; the remaining effects require human review and remedy.’

This is much more useful than ‘An error occurred.’ It brings the problem, the behaviour and the responsibility into view together.

The chapter's conclusion

A strong system does more than perform the right actions quickly.

A strong system:

• notices what is wrong,
• stops ongoing harm,
• reduces the scope of impact,
• returns to the last safe state,
• verifies its claim of success independently,
• preserves evidence,
• informs people,
• allows challenges,
• seeks to remedy harm,
• updates the behavioural contract.

Error-free operation cannot be achieved by a promise alone. Recoverability, however, is a capability that can be designed.

That is why GBO is more than an optimisation field. It seeks not to increase behaviour but to make behaviour responsible. An agent may find the right person, understand a real capability, make a suitable choice and act with valid authority. Things can still go wrong. That fact does not make the system a failure. Having no recovery route makes it unreliable.

The NOMOS Recovery and Remedy Contract therefore states: every significant action should be designed not only with a plan for success, but also with plans for stopping, reversal, challenge and remedy.

At the end of Part II, we now have the entire core of the behavioural contract:

Without identity, there is no correct counterpart. Without capability, there is no real promise. Without suitability, there is no right selection. Without authority, there is no legitimate action. Without recovery, there is no sustainable trust.

But these contracts will not change the world if they remain only in a book. Companies, institutions, products and agent systems need to turn them into everyday working practices. In the next part, we move from explaining the concept to putting it into practice.

Our first question will be:

How does an organisation become agent-ready?

Because being agent-ready means more than publishing an API or putting an ‘AI-ready’ label on a website.

An agent-ready organisation has:

• resolved its identity,
• bounded its capabilities,
• explained its suitability conditions,
• established its authority map,
• tested its recovery route for failure.

These conditions define its readiness.

Opening a channel through which a machine can act does not make that action responsible. Appropriate authority, oversight and reversal limits require organisational design.