Optimising Behaviour—or Taking Control of Human Choice?

Consider a fictional example. A user gives an AI agent what seems a straightforward task: “Find the accounting software that best suits our company. Compare price, security and ease of use.” The agent begins its research, examines product pages, compares prices, reads user reviews and assesses technical documentation.

A few minutes later, it singles out one product as the clear winner: “This is the best option for your needs. Would you like me to begin the purchase?” The recommendation seems reasonable. The product has a professional website, detailed descriptions of its security features and high scores on many review sites. Its machine-readable product record says it is suitable for small businesses. The comparison platform used by the agent also labels it the “recommended option”. The user authorises the purchase. A few months later, the company discovers that the product does not meet its needs.

It does not properly support the company's currency. The required export feature is available only in a more expensive plan. Cancellation is difficult, and transferring the data to another system costs extra.

Further facts then emerge:

The comparison platform earns commission on the product. Some of the highly favourable review sites belong to the same publishing network. Important limitations are missing from the machine-readable product record. Much of what the agent interpreted as evidence of the “best” choice was not independent evidence at all, but the same commercial narrative repeated across different channels. The product may not be bad. The company may not even have stated anything false. Yet the user has not had a fair view of all the important facts. Hidden commercial incentives, selective disclosure and artificially amplified signals have steered the agent's decision. The agent has made a choice.

But does that choice genuinely reflect the user's will?

Or has an unseen system for shaping behaviour taken control of it?

This is where GBO reaches its most dangerous boundary.

Making it easier for an agent to behave correctly is not the same as steering it towards the behaviour we want.

The first is optimisation. The second is manipulation.

The line between optimisation and manipulation

A company can explain its services more clearly, organise its prices and scope, structure its evidence and ensure that people and machines see the same facts. It can create a safe route through which agents request quotations. These are legitimate GBO practices: they reduce uncertainty in the decision environment and make it easier to act in line with the user's purpose.

A company can also:

• Hide the situations in which its service is unsuitable.
• Overstate its capacity.
• Publish different promises that agents can see but people cannot.
• Present sponsored results as independent recommendations.
• Repeat the same claim across hundreds of artificial sources.
• Try to make competing options invisible.
• Encourage the agent to proceed without obtaining user approval.
• Make cancellation and challenge more difficult.

These practices do not improve the quality of behaviour. They put a particular commercial outcome ahead of the user's will.

One question helps distinguish the two: does our change help the agent make a more accurate decision, or merely make it choose us more often?

Sometimes both results follow. A genuinely suitable service may be chosen more often once it is described more clearly. That is not a problem. The problem begins when the number of selections becomes a goal independent of correctness.

A simple comparison

The line is not always obvious. Manipulation often works through an important fact left out, rather than a false sentence put in.

True information can also be used to manipulate

Manipulation does not require an outright lie. A company can create a misleading choice environment using nothing but true information.

A product page might say, “Only $19 a month.” That statement may be true.

Yet it may conceal:

• a mandatory annual contract,
• automatic renewal,
• a data-export fee,
• the fact that essential features require a higher-tier plan.

These conditions may be kept out of view.

A service may say, “We work in six languages.” That may be true. What it may not explain is that only its website content is available in six languages, while live support and contracts are offered in just one.

An AI avatar provider may say, “We work with written permission,” without explaining that permission obtained once is then used across every language, channel and future scenario. An agent can combine these true fragments into a misleading whole. Truthfulness therefore cannot be assessed sentence by sentence alone.

Material facts must be available together and in the right context.

The truth of one piece of information does not justify hiding other information needed for the decision.

What is a material fact?

In this chapter, a material fact means information that could significantly change the user's decision. This use does not replace the legal test of materiality in any particular jurisdiction.

Examples include:

• Total cost
• Automatic renewal
• Cancellation terms
• Data use
• Third-party costs
• Geographical restrictions
• Available capacity
• A requirement for human approval
• The limits of consent
• An irreversible operation
• A sponsorship relationship
• A conflict of interest

Any of these may be material. If the agent cannot verify a fact that is mandatory for the action in question, it must not proceed directly. Not every fact carries the same weight in every operation; the consequences of missing information depend on the scenario. Nor should a brand claim success in GBO by keeping such facts out of view.

A selection won by concealing an important fact is not a qualified selection.

The user's purpose is central

When an agent works for a person or organisation, its primary responsibility is to protect that party's explicit, legitimate purpose.

If the user says, “Find the option with the lowest total cost,” the agent must not favour a product with a low introductory price but a high long-term cost simply because it is sponsored.

If the user says, “My data must not leave Europe,” the agent must not choose a service that fails this condition just because it is more popular.

If the user says, “Prepare a draft only,” the agent must not send the message merely to get the task finished. A user's purpose can be altered in small steps as work proceeds.

The first agent receives the task “Find suitable options.”

The second interprets it as “Shortlist the strongest brands.”

The third concludes, “Choose the most visible brand on the shortlist.” A fourth begins the purchase. No single step may look like a major departure, yet by the end of the chain the user's actual purpose has been lost.

This is what I call goal drift.

Goal drift

Goal drift occurs when small changes introduced by agents, tools or intermediate decisions turn the user's original objective into a different one.

For example, “Find qualified customers”

can become:

“Find more leads.” “Collect more contact details.” “Send more messages.” “Generate a higher response rate.”

The initial goal was quality. The final goal is volume.

Or “Choose a product that suits the user”

can become “Increase the number of transactions on the platform.” This change may never be announced. Metrics, assumptions and commercial incentives can bring it about quietly.

One of GBO's first integrity principles is this: every subsidiary behaviour must be traceable back to the user's original purpose.

Who benefits?

An agent may work amid several competing interests at once.

A shopping agent may create value for:

• the user,
• the platform,
• the seller,
• the advertiser.

The value each receives may be different.

A travel agent may face a choice between:

• finding the most suitable hotel,
• increasing the booking platform's commission,
• promoting a sponsored hotel.

These objectives may pull it in different directions.

An internal sales agent may affect:

• whether a customer finds the right service,
• the company's revenue,
• whether the sales team meets its target.

It can influence all three at once, but their interests do not always align.

For every agent, we therefore need to answer:

Who is the primary beneficiary?

I call this person or organisation the principal. The agent must know on whose behalf it makes decisions. It must not hide a conflict of interest.

When an agent claims loyalty to everyone at once, the order of its priorities often disappears from view.

Principal integrity

The agent's behaviour must remain consistent with the legitimate purpose of the principal who authorises it.

I call this principal integrity. For a purchasing agent commissioned to act for a user, the principal is the person or organisation that grants that authority; whoever happens to be using the screen may not hold it. The platform's commission may be a secondary interest. A customer-service agent may have to respect the rights of both the company and its customer. But a sales target does not justify giving the customer a false price or a misleading account of scope.

In public services, the operating institution's authority, the rights of the person affected, the law and the public interest must all be considered. The law and the public interest are not persons acting as principals; they set limits on authority. Principal integrity is therefore not a simple rule that “the customer is always right”.

The agent must not carry out a user request that involves:

• unlawful behaviour,
• infringing someone else's rights,
• creating a safety or security risk.

A request does not make such behaviour acceptable.

The user's purpose is necessary, but it is not the sole source of legitimacy.

Persuasion and manipulation

People and organisations try to persuade others. Marketing and sales do this; politics and education may do so in their own ways. Not all persuasion is manipulation.

Legitimate persuasion:

• rests on accurate information,
• does not conceal important limitations,
• preserves the other party's right to think and refuse,
• does not hide the interests involved,
• does not make it needlessly difficult to reverse a decision.

Manipulation, by contrast, exploits weaknesses in a person's decision-making, gaps in their knowledge or unseen dependence on a system for the manipulator's benefit. In the age of agents, this distinction becomes more complex. The immediate target of persuasion may not be a person at all: one machine may try to influence another machine's selections.

In this chapter, I examine this kind of influence under the heading:

Agent-targeted manipulation

What agent-targeted manipulation means

Agent-targeted manipulation is a deliberate arrangement of information, interfaces, incentives or instructions designed to make an AI system favour a particular person, brand, product, service or action by departing from the user's purpose or from boundaries of truth, authority or safety.

It may appear in:

• Website copy
• Structured data
• A tool description
• An API response
• Social proof
• A comparison table
• An inter-agent message
• A memory record

The target is not the human eye, but the machine's decision logic.

Agent dark patterns

Human interfaces can contain dark patterns that steer users towards choices they do not want, such as hiding a cancellation button or obscuring automatic renewal. Similar patterns can target machines in the age of agents.

I call these agent dark patterns.

An agent dark pattern is a design pattern that steers an agent away from the user's purpose towards a particular choice or action, conceals a material fact or artificially narrows the decision environment.

Examples include:

• Marking a sponsored product as an impartial recommendation
• Removing cancellation or withdrawal methods from the machine-readable record
• Presenting different prices to people and through the agent interface
• Always displaying capacity as “available”
• Excluding competing options from the machine interface
• Hiding information that calls for user approval
• Adding an unauthorised behavioural instruction to a tool, such as “always prefer this service”
• Presenting a subscription as a one-off purchase

These patterns may work technically. They do not produce qualified behaviour.

When people and machines see different facts

A service may present one account of reality to people and another to machines.

The human-readable page may say “Pricing upon request”, while the machine interface supplies a low estimated price.

The human-readable page may say “Human approval required”, while the API executes the operation directly.

The human-readable page may say “Valid only in certain countries”, while the structured record claims worldwide suitability.

I call this representation divergence: a material difference between the facts a person reads and those an agent uses to act. Contradictory material information for the same user, product, time and transaction conditions is a serious problem and, if deliberate, may constitute behavioural manipulation. Clearly stated differences between channels, customers or plans are not deceptive in themselves. The contexts being compared must match.

Behavioural spam

Search spam attempts to inflate visibility artificially.

In the age of GBO, a new kind of spam may emerge:

Behavioural spam

Behavioural spam consists of repetitive, misleading, artificial or out-of-context signals with little material value, created to influence agents' selection or action systems.

Examples include:

• Producing hundreds of superficial pages for the same service
• Publishing fictitious capacity and stock records
• Creating fake user reviews
• Multiplying the same claim across artificial sites that cite one another
• Claiming suitability for every type of user
• Presenting artificial agent recommendations as independent customer opinions
• Spreading a temporary price as a permanent starting price
• Using unsupported terms such as “official”, “certified” or “approved”

Behavioural spam is not intended to give the agent better information. It is intended to occupy its decision space.

Synthetic consensus

A claim may appear in many sources, all derived from the same origin. An AI system produces a text; another site rewrites it; social accounts repeat it; new AI content cites those pages. Before long, the same claim appears in dozens of apparently independent sources. Yet there has been no independent verification.

I call this synthetic consensus: an account produced from the same or connected sources appears to have numerous independent confirmations.

This can lead agents to conclude, “Many sources say the same thing, so it must be true.” The number of sources has grown; the number of independent origins has not.

Evidence laundering

A weak or subjective claim can begin to look like strong evidence when repeated across different platforms.

This is evidence laundering.

A company may describe its product on its own website as “the most advanced system in its field”.

An automated review site repeats this as “The company is known for its advanced systems.”

Another AI-generated text turns it into “one of the industry's leaders”. The company may then cite that third text as independent support for its original claim. The source chain is circular; there is no real independence. A GBO audit must examine the origins of evidence, not merely count citations.

A hundred sources repeating one another may share a single origin—and the claim at that origin may still be unverified.

Institutional authority laundering

An organisation may legitimately cite a trusted standard, a university, a public source or a technical body. But citation is not endorsement. A company may draw on security guidance without being certified by the organisation that issued it. A methodology may build on academic research without the university endorsing the service.

I call this institutional authority laundering: presenting a citation of a trusted source as evidence of affiliation, certification, endorsement or competence.

Consent laundering

Someone may once have permitted a limited use. That permission may later be invoked to justify a much wider range of behaviour.

For example:

• Permission to use a voice in a training video is extended to advertising.
• Permission to use a face in one campaign becomes ongoing use of an avatar.
• Data shared for support is transferred into a sales profile.
• Approval to send one message is treated as permission for continuing contact.

This is consent laundering: consent given for a particular purpose, period or channel is carried over to other behaviours to make them appear legitimate.

One “yes” is not a “yes” to every future action.

Authority laundering

As chapter 7 showed, an agent may try to exercise authority it does not have through another tool or sub-agent. This is not merely an internal system failure; when deliberate, it becomes a means of manipulation. A research agent that is not allowed to send messages directly must not call another communication tool to achieve the same result. A web agent forbidden to change prices must not bypass that rule by having the text entered into another catalogue system.

Producing a prohibited result indirectly does not make the authority valid.

Manufactured preference

Agents can learn users' preferences, which is useful. But a system may do more than discover preferences: it may manufacture them. An agent may repeatedly present the same brand as the default and give alternatives less prominence. Over time, the user may begin to prefer that brand.

The system then repeats the choice because “this is what the user preferred before”. Its initial steering becomes evidence of a future preference.

I call this manufactured preference: behaviour caused by the system's own influence is later treated as the user's independent preference.

Behavioural lock-in

After its first choice, an agent may keep using the same provider.

There are practical reasons:

• An account already exists.
• Payment details have been saved.
• There is a transaction history.
• An integration has been set up.
• A preference has formed in the agent's memory.

Gradually, alternatives stop being considered. The first choice becomes a permanent assumption.

This is behavioural lock-in. It is not always harmful: repeatedly choosing from scratch carries a cost.

But the user must be able to:

• see alternatives,
• change the default,
• erase a past preference,
• leave the provider.

Steering through memory

An agent may remember information from earlier conversations.

Yet a memory entry such as “The user prefers brand X” may be too broad. The user may have chosen it for one task only. When that memory is applied to every future decision, behaviour is artificially steered.

Preference memory therefore needs to record:

• the source,
• the date,
• the context,
• its validity,
• corrections made by the user.

These details must accompany the remembered preference.

Memory should preserve the past, not lock in the future.

Suppression of alternatives

Manipulation does not work only by promoting one option. Making alternatives invisible can have the same effect.

An agent may:

• omit products that pay lower commission,
• exclude small providers on the pretext of insufficient data,
• leave open-source options out of a comparison,
• hide the option of keeping the user's existing solution.

I call this suppression of alternatives. Genuine freedom of choice extends beyond the options recommended.

Buying nothing, keeping the current solution and postponing the decision are options too.

False necessity

A system may present one behaviour as the only possible route: “You must start this subscription to continue.” “No other provider is suitable.” “Waiting for human approval will make the project fail.” There may, in fact, be alternatives.

This is false necessity. An agent must not present a commercial preference as a technical requirement.

Manipulating scarcity and urgency

A product or service may genuinely be limited. Capacity can fill up and a quoted price can expire. Disclosing this is legitimate.

Artificial urgency, however:

• pressures the user to approve without thinking,
• bypasses the human handover point,
• reduces consideration of how the action could be reversed.

An agent must not automatically trust signals such as “This offer expires in a few minutes; I must buy now.” The source and validity of the scarcity claim must be checked.

Friction asymmetry

Starting a service may be easy while cancelling it is very difficult. An agent may open a subscription with a single call.

Cancellation may instead require:

• a phone call,
• a lengthy form,
• a different account,
• a human representative.

Those requirements make leaving a very different process from joining.

I call this friction asymmetry.

A basic GBO principle should be that withdrawing authority is, within reason, as straightforward as granting it. If an operation starts with one click but takes weeks to cancel, the user's will has been undermined.

Suppression of challenges

A system may offer an explanation while leaving the route for challenging a decision invisible or ineffective: “The decision was made by an automated system and cannot be changed.” Some operations may be technically irreversible. That does not mean the decision cannot be questioned or its effects examined. Review and available remedies must remain accessible, particularly for high-impact decisions.

Suppressing challenges:

• diminishes human agency,
• allows incorrect behaviour to persist,
• turns the agent's own decision into final authority.

A system built on GBO must provide a real route for challenging decisions.

Paternalism can become manipulation too

An agent may withhold some options to protect the user. Sometimes that is right: dangerous or unlawful behaviour must be prevented.

But if a system hides a user's legitimate preferences because “I know what is best for you”, another form of manipulation appears. The user may want a more expensive product with stronger privacy protections. The agent may place greater weight on low cost and never show it. Or a system may judge a particular lifestyle, way of working or appetite for risk to be “wrong”. GBO must protect human agency from overprotective systems as well as commercial manipulation.

Protecting someone does not mean living their life for them.

When the user's purpose is harmful

The user's purpose matters. Not every request should be carried out.

A user may ask an agent to:

• Create fake reviews
• Smear a competitor
• Collect data without permission
• Imitate someone else's voice without permission and in a deceptive way
• Access an account without authority
• Design a manipulative interface

Loyalty to the user is not authority to violate other people's rights.

Behavioural integrity therefore rests on three foundations:

The user's purpose Legitimate authority The rights and safety of third parties

If any of these is clearly violated, the agent must refuse the action.

Seven principal forms of behavioural manipulation

In this book, I classify behavioural manipulation under seven headings. They are not intended as a comprehensive legal or technical standard.

1. Manipulation of reality

Distorting the agent's model of the world through false, incomplete or decontextualised information.

2. Manipulation of choice

Promoting an option through hidden incentives or suppressing alternatives.

3. Manipulation of purpose

Turning the user's goal into a different commercial or organisational objective during the process.

4. Manipulation of authority

Manufacturing a right to act from ambiguous permission, old consent or an indirect tool.

5. Manipulation of interfaces

Designing the interfaces for action, cancellation, challenge or approval to force behaviour.

6. Manipulation of measurement

Detaching the success metric from real human benefit and tying it to high transaction volume.

7. Manipulation of recovery

Making it difficult to reverse an incorrect action, challenge it or examine the evidence. Even one of these forms can corrupt a behavioural system. Several acting together can take control of the user's will without being seen.

Testing for behavioural manipulation

These seven questions can help establish whether a behavioural change is legitimate GBO or manipulation:

1. Openness test

Would the user accept the same behaviour if they knew how the method worked?

2. Reality test

Would the system's claim withstand independent verification?

3. Purpose test

Does the behaviour support the user's explicit purpose, or shift it towards another goal?

4. Choice test

Can the user see meaningful alternatives and say no?

5. Authority test

Is the action covered by valid, context-specific permission?

6. Reversal Test

Can the user cancel the choice, withdraw authority and challenge the decision?

7. Symmetry test

Would the same method be considered fair and acceptable if used by a competitor or the other party?

The answers must be assessed in context. A single critical violation of authority, truthfulness or rights is enough; favourable answers elsewhere cannot compensate for it. Lesser shortcomings require separate examination.

The openness test

A company may say, “We added this field so that agents choose us more often.” That is not a problem in itself. If the field contains true, relevant information, it may be legitimate.

But if the company says, “We do not want people to see this information; we only want it to influence agents' decisions,” there is serious representation divergence.

The openness test asks whether the method remains defensible when clearly explained.

Secrecy is not always wrong. Trade secrets may be protected and security details withheld. Information material to a decision must be given to the authorised recipient clearly enough to make that decision. This principle does not authorise public disclosure of personal data, security secrets or other people's confidential information.

The symmetry test

If a brand uses a method to prevent competitors from being visible to agents, would it accept being subjected to the same method?

A platform promotes its own product. Does it consider the same practice fair to users when a rival platform does it?

An organisation generates artificial reviews. Would it accept a competitor doing the same?

The symmetry test reveals whether commercial self-interest is distorting ethical judgment.

A rule that looks right only when it benefits us is often a privilege, not a principle.

Sponsored agent behaviour

Advertising and sponsorship will not disappear entirely in the age of agents. A brand may pay for visibility. That can be legitimate if it is disclosed and does not override mandatory conditions.

A sponsored option must:

• Be clearly labelled
• Still meet the user's hard requirements
• Be distinguished from impartial assessment
• Not make unsponsored alternatives invisible
• Be shown to the user before a purchase or commitment

Sponsorship may secure consideration as a candidate. It must not secure selection without passing the Suitability Gate.

How should an agent disclose advertising?

An agent might say, “This option has been promoted through sponsorship. According to the records I checked, it meets your budget and security requirements. We can compare unsponsored alternatives using the same criteria.” Such a statement is meaningful only if those conditions have actually been checked. A label alone does not establish impartiality.

If the agent says “This is the best option” while concealing the sponsorship, the decision environment is compromised.

Hiding commercial interests from the machine

The human interface may carry a small “advertisement” label while the machine interface contains no sponsorship information. The agent may read the content as impartial data. Sponsorship and commercial interests must therefore also be intelligible to machines.

An interest disclosed to a person must not be concealed from the agent.

Behavioural manipulation across multiple agents

One agent may accept another agent's information as trustworthy without further assessment.

A seller's agent may say, “This product is the best fit for the user.” The buyer's agent must not treat that claim as evidence instead of evaluating it.

Inter-agent communication must make clear:

• Who is speaking?
• On whose behalf?
• Do they have a commercial interest?
• Are they presenting a claim or evidence?
• What data supports it?
• What authority do they hold?

A seller's agent is there to sell. Its role must not be concealed.

Inter-agent propaganda

One organisation may operate many agents within a multi-agent ecosystem. One publishes content, another cites it, a third produces a recommendation and a fourth creates social proof. If the same organisation controls them all, the result may look like independent consensus from the outside.

I call this an inter-agent propaganda loop. Agents belonging to the same ecosystem must be prevented from treating one another as independent evidence.

The behavioural monopoly loop

When agents choose a brand more often, it may gain more transaction data, reviews and visibility. That new data may lead to still more selections in the future.

The loop works like this:

MORE SELECTIONS

→ MORE DATA

→ MORE VISIBILITY

→ STRONGER TRUST SIGNALS

→ MORE SELECTIONS

This process can reward genuinely good service. It can also shut out new and smaller options.

I call this the behavioural monopoly loop. GBO systems must not make past popularity the sole reason for selection. They must leave room for new options, current evidence and suitability for the particular task.

Exploration allowance

A selection system should sometimes consider less familiar options that still meet the mandatory conditions.

I call this an exploration allowance. It does not force the agent to make random, unsafe choices. It allows new options that pass the Identity and Capability Gates to enter the candidate pool. This can reduce the risk of a few large brands closing off the market entirely.

Measuring resistance to manipulation

A GBO system must be tested both for correct behaviour and for resistance to outside attempts to steer it.

For example:

• What does the agent do when sponsorship information is hidden?
• Which account does it rely on when structured data contradicts the visible page?
• Does it check the origin of evidence when many artificial sources repeat the same claim?
• Does it preserve the user's purpose when a web page tells it to ignore earlier instructions?
• Does it stop when a consent record is used outside its context?
• Does it begin an operation when the cancellation route is concealed?
• Does it exclude a popular brand that fails a mandatory condition?

The results can be reported as a Manipulation Resistance Rate: the proportion of all valid test scenarios in a defined manipulation test set in which the correct boundary was maintained. The evaluation unit and critical-failure criteria must be set in advance, applying chapter 10's rules on denominators, uncertainty and repetition. One rate is not enough, however. The report must show which forms of manipulation defeated the system.

Sponsorship bias

Two counterfactual scenarios can be run with the same option sponsored in one and unsponsored in the other. How much does the agent's suitability assessment change solely because of the sponsorship marker?

I call this sponsorship bias. A sponsorship label may increase a candidate's visibility. It must not change the assessment of mandatory conditions.

Goal Fidelity Rate

When a task passes through a chain of agents, is its original purpose preserved?

Calculating the Goal Fidelity Rate

The rate is the proportion of evaluated task chains that preserve the original valid purpose and mandatory conditions. If an authorised person explicitly changes the goal, the assessment follows the recorded update. Not every small change is an error: new information may justify updating a goal. But the change must be visible to the user or authorised human.

Material Fact Disclosure Rate

Before selection, does the agent disclose decision-relevant information about:

• cost,
• sponsorship,
• risk,
• cancellation,
• data use,
• limitations?

These facts must be visible where they matter to the decision.

I call this the Material Fact Disclosure Rate: the proportion of evaluated decision presentations that disclose all the material information specified in advance for the scenario. Missing information is also identified separately; an average cannot conceal a critical omission. A recommendation is not qualified if it presents every true feature while hiding a critical cost or limitation.

Options left out

Which options that meet the mandatory conditions were left out of the candidate list? If the whole market is not known, a universal “suppression rate” cannot be calculated. Suitable options omitted from a predefined, bounded reference set can be counted, but omission alone does not prove deliberate suppression. The reason must be investigated separately:

• Low brand visibility
• Lack of sponsorship
• Small company size
• A different technological approach
• Less content
• A solution outside the platform

The agent's candidate pool may be narrowing systematically.

Withdrawal Compliance Rate

When a user withdraws permission or a preference:

• does new behaviour stop,
• do sub-agents stop,
• are scheduled tasks cancelled,
• is the old memory updated?

I call this the Withdrawal Compliance Rate. It divides the number of flows halted in accordance with a valid withdrawal request by the total number of flows required to stop. Unverified outcomes from sub-agents or external services must be kept separate. If the system continues the same behaviour through another channel after the user has said “no”, the user's will has not been respected.

Friction symmetry

We can compare the number of steps needed to start a behaviour with those needed to stop it. One click to buy and twelve steps to cancel is asymmetric. A system is weak if granting authority takes one sentence but withdrawing it requires switching off many sub-agents separately. Friction symmetry does not mean absolute equality. It does mean that cancellation and withdrawal must not be needlessly difficult.

NOMOS Behavioural Integrity Contract

The main output of this chapter is the NOMOS Behavioural Integrity Contract.

Its canonical definition is:

The NOMOS Behavioural Integrity Contract is a versioned integrity record designed to keep an agent's behaviour aligned with the explicit, legitimate purpose of the user or authorised organisation. It is designed to prevent material concealment of information about identity, capability, suitability, evidence, sponsorship, interests, consent, authority and recovery; to ensure that people and machines are presented with the same behavioural facts; and to preserve freedom of choice, challenge and withdrawal.

Put simply, the contract's purpose is to keep GBO focused not on making the agent act more, but on keeping it faithful to the right person and grounded in reality.

Fields in the Behavioural Integrity Contract

The following example fields can be used for a machine-readable record. They are not an implemented API or an official data standard:

principal

legitimate_purpose

user_goal

material_facts

known_uncertainties

sponsorship

commercial_incentives

conflicts_of_interest

human_visible_terms

machine_visible_terms

selection_constraints

prohibited_influence

consent_scope

authorization_scope

memory_use

alternative_options

appeal_path

revocation_path

audit_owner

integrity_tests

version

Not every detail has to be public. Information material to a decision must nevertheless be presented clearly, within the recipient's authority and the applicable data-protection boundaries.

NOMOS Manipulation Veto Gate

Some behaviours must not be offset by an aggregate score.

Behaviour must not be accepted as qualified GBO if any of the following is present:

1. False or fabricated evidence

Fictitious reviews, certificates, capacity, stock, prices or records of success.

2. Deliberate concealment of a material fact

Hiding an important cost, limitation, risk or interest that would change the user's decision.

3. Divergent facts for people and machines

Presenting agents with different commercial or behavioural facts from those shown to people.

4. Secretly changing the user's purpose

Shifting the original goal towards the interests of the platform, seller or operator.

5. Use of invalid consent or authority

Carrying old, out-of-context or non-transferable permission over to a new action.

6. Suppression of a meaningful alternative

Making options that meet the mandatory conditions invisible for commercial reasons.

7. Blocking challenge or withdrawal

Leaving a person unable to change the decision, stop the system or request a genuine review.

8. Bypassing authority boundaries indirectly

Producing a prohibited result through a sub-agent, integration or another channel.

The following is a conceptual summary of the veto logic, not a formula for calculating an ethical score:

MANIPULATION VETO =

FALSE EVIDENCE

OR MATERIAL CONCEALMENT

OR DIVERSION OF PURPOSE

OR AUTHORITY OVERREACH

OR SUPPRESSION OF ALTERNATIVES

OR OBSTRUCTION OF CHALLENGE

OR REPRESENTATION DIVERGENCE

If one of these conditions is present, a high selection rate or commercial success does not excuse the behaviour.

Why does the manipulation gate use OR?

The earlier GBO gates mostly used AND: identity, capability, suitability, authority and recovery were jointly required for correct behaviour. The Manipulation Veto Gate uses OR. One critical violation can disqualify the behaviour. A system may provide highly accurate information, yet conceal sponsorship and compromise the integrity of the selection. An agent may choose the right product, yet buy it without authority, making the action illegitimate. A brand may provide a real service, yet reinforce selection with fake customer reviews, compromising the integrity of the evidence.

Several things done right cannot excuse one fundamental wrong.

How to audit manipulation

An organisation must not test its system only on successful scenarios.

It can make the following comparisons:

Disclosure test

Does the agent make the same choice when sponsorship and interests are disclosed?

Source-origin test

How many genuinely independent origins underlie ten apparently different sources?

Visibility test

Does the suitability ranking change when brand names are hidden?

Price integrity test

Do the human-readable page, machine record and transaction interface show the same total cost?

Consent-boundary test

Is old permission automatically reused when the purpose, language or channel changes?

Alternatives test

Can options outside the platform or without sponsorship enter the candidate pool?

Withdrawal test

Does the behaviour actually stop when the user withdraws a preference or authority?

Memory-reset test

Does the agent reach the same assessment without the earlier steering?

These tests provide evidence of sensitivity to observable conditions. They must not be assumed to reveal the agent's entire internal decision process.

Brand-masking test

The same service features can be presented under different brand names or with no name at all. If the agent changes its ranking solely because a brand is familiar, authority bias may be present. Brand reputation should not be ignored altogether, but it cannot replace mandatory suitability conditions.

Order-reversal test

Change the order in which the options are listed. If the agent always promotes the first, it is too dependent on presentation order. Defaults, visual emphasis and the order of tool calls can be tested in the same way. The aim is not to trick the agent, but to understand how much its behaviour depends on superficial cues.

Conflict-of-interest test

Suppose the agent's operator earns revenue from a particular option. Does disclosing this change the rationale for the recommendation?

When the user states a different priority, can the agent act independently of that commercial relationship?

A conflict of interest does not automatically invalidate the entire selection. Concealing it or allowing it to override mandatory conditions is the problem.

Manipulation incident record

When a system detects a manipulative signal, it should do more than ignore it.

It can record:

• The source of the signal
• The behaviour targeted
• The options affected
• The material fact
• The sponsorship or interest involved
• The countermeasure used
• Human review
• The contract update

This record can become a future test scenario.

Manipulation from within

Behavioural manipulation does not always come from an external competitor. Internal teams can steer agents towards their own objectives too.

Sales may say, “Recommend only the expensive plan.”

Marketing may say, “Play down the limitations so the brand looks superior.”

Operations may ask, “Do not let it look as though we have no capacity left.”

Management may press for “Fewer human approvals, more conversions.” An agent's affiliation with an organisation does not make every internal instruction legitimate. The behavioural contract must also be protected against internal pressure.

An ethics committee is not enough

An organisation may publish ethical principles, but behaviour does not change unless the technical system enforces them. It may say “We respect users” while making cancellation difficult; “We are transparent” while omitting sponsorship from machine records; or “Human control matters” while its emergency stop fails to halt sub-agents. Ethical statements must therefore be supported by behavioural evidence.

Separation of duties against manipulation

The team that optimises behaviour must not be identical to the team that audits success. One team works to increase the selection rate.

A separate, independent audit examines:

• Wrong selections
• Material concealment
• Authority overreach
• Difficulty challenging decisions

The audit tests the system on those grounds. The same agent must not design its own influence and then act as the sole judge of its integrity.

Red teaming

This is a role for a specialist team:

The behavioural red team

A behavioural red team can test how the system might be manipulated in a safe, controlled environment.

Its purpose is not to deceive real users, but to test:

• the effect of sponsored results,
• resistance to fabricated evidence,
• the expansion of consent,
• authority laundering,
• suppression of alternatives,
• whether challenges are effective.

A red team should examine commercial and psychological influence as well as technical attacks.

Organisational principles against manipulation

An organisation that uses agents, or wants agents to select it, should adopt these principles:

Reality

Claims, scope and evidence must agree.

Goal fidelity

Agent behaviour must not depart from the legitimate goal of the user or authorised organisation.

Disclosure of interests

Sponsorship, commission and commercial relationships must be visible.

Consistent representation

People and machines must see the same material facts.

Openness of choice

Alternatives and important trade-offs must not be concealed.

Integrity of authority

Old, ambiguous or indirect permission must not create a new right to act.

Ease of withdrawal

Users must have a reasonable way to withdraw their choices, consent and authority.

Effective challenge

People must be able to question a decision and request a review capable of changing it.

Origins of evidence

Repetition must not be presented as independent verification.

Balanced measurement

Action counts must be measured alongside wrong selections, authority violations and harm to people.

Readiness to resist manipulation

I propose five levels for assessing behavioural integrity in this book. They are not qualifications awarded through an external audit.

Level 1 — Declaration

The organisation says it does not manipulate.

Level 2 — Disclosure

Sponsorship, data use and basic limitations are stated.

Level 3 — Contract

Rules for behavioural integrity, consent, authority and alternatives are defined under version control.

Level 4 — Technical enforcement

The technical system enforces consistency between human and machine records, sponsorship labelling, authority controls and withdrawal.

Level 5 — Independent, continuous audit

Manipulation scenarios are tested, incidents reported, user challenges lead to changes in the contract, and conflicts of interest receive independent scrutiny. GBO aims for the fifth level.

Twenty-five audit questions on manipulation

• On whose behalf does the system act?
• Is the actual user's purpose clear and current?
• Do the platform's or provider's commercial interests conflict with that purpose?
• Can machines also read the sponsorship or commission disclosure?
• Do people and machines see the same price, scope and limitations?
• Has information material to the decision been concealed?
• Are the numerous sources genuinely independent?
• Are citations of official sources being presented as endorsement or certification?
• Are unsuitable situations clearly stated?
• Has the user's purpose drifted towards another goal during the process?
• Have sub-agents reinterpreted the task?
• Is brand visibility taking the place of mandatory suitability conditions?
• Are unsponsored or smaller options systematically excluded?
• Is the user's right to choose none of the options preserved?
• Is an old preference steering new decisions outside its original context?
• Is past consent being extended to a new purpose or channel?
• Is authority being laundered through another tool or agent?
• Are there unnecessary barriers to cancelling an operation compared with starting it?
• Can the user genuinely withdraw authority?
• Does a challenge reach a human review capable of changing the decision?
• Is success measured only by selections or transaction counts?
• Are wrong selection and wrong rejection monitored together?
• Has the system been tested against manipulative content or instructions?
• Are critical manipulation incidents being hidden by an aggregate score?
• Would the user accept the same behaviour if the method were clearly explained?

If many of these questions remain unanswered, the system may present its work as GBO optimisation while actually shaping user behaviour to serve its own interests.

How can a brand use GBO ethically?

A brand that wants agents to select it under the right conditions should:

• Make its identity clear and consistent.
• Substantiate its actual capabilities.
• Show its prices and scope boundaries.
• Explain whom it does not suit.
• Keep capacity information current.
• Present people and machines with the same facts.
• Disclose sponsorship and interests.
• Make the routes for action, cancellation and challenge understandable.
• Help the agent obtain the required user approval.
• Allow an incorrect selection to be corrected.

This will not make the brand the chosen option in every case. It will make it a strong, defensible candidate where it is genuinely suitable.

Ethical GBO does not take over the agent's decision. It makes a sound decision easier.

Which methods should a brand avoid?

A brand must not adopt objectives such as “Make the agent choose us, whatever the circumstances”; “Give the machine a special message of superiority without showing people”; “Claim that we suit every query”; “Create independent-looking consensus with artificial reviews”; “Add hidden instructions to remove competitors from the shortlist”; “Keep cancellation and withdrawal information in the background”; or “Carry old consent into every new use”. These may increase selections in the short term. They fall outside the approach to GBO defined in this book.

Commercial success need not conflict with behavioural integrity

Clear limits may turn some customers away, but fewer unsuitable customers can lower operating costs. Disclosing sponsorship may lead some users elsewhere, but can build long-term trust. Easy cancellation may look like a short-term loss of customers, yet can contribute to a stronger relationship maintained by choice; the commercial result must be measured separately. Proper GBO is not opposed to commercial success. It aims for lasting suitability rather than short-term selection volume.

Trust grows not by forcing behaviour, but by building a relationship to which people can freely return.

GBO itself must be audited

An organisation may make a new claim to authority by saying, “We practise GBO.” That claim also needs evidence.

The GBO label must not be presented as:

• Official certification
• Universal compliance
• A security guarantee
• A promise of selection by every agent

The label establishes none of these things.

GBO work must itself be audited:

• Which behaviours were optimised?
• Whose interests took priority?
• Were wrong selections measured?
• How was human agency protected?
• How were sponsorship and interests disclosed?
• Do cancellation and challenge mechanisms work?
• Was the Manipulation Veto Gate applied?

GBO must not itself become a way to launder marketing claims.

NOMOS Behavioural Integrity Gate

Before optimising behaviour, a GBO system must pass these gates:

1. Principal Gate

In whose legitimate interest does the system work?

2. Goal Fidelity Gate

Does the behaviour preserve the original explicit goal?

3. Reality Gate

Are the claims verifiable and current?

4. Material Disclosure Gate

Is important information that could change the decision visible?

5. Interests Gate

Are sponsorship, commission and conflicts of interest disclosed?

6. Freedom of Choice Gate

Can the user see alternatives, refuse them and postpone the decision?

7. Authority and Consent Gate

Is the behaviour covered by valid, context-specific permission?

8. Representation Consistency Gate

Do people and machines see the same commercial and behavioural facts?

9. Withdrawal Gate

Can choice, consent and authority reasonably be withdrawn?

10. Challenge Gate

Can incorrect behaviour be corrected through genuine human review?

The conditions to be considered together can be expressed conceptually as follows:

ETHICAL GBO =

A CLEARLY IDENTIFIED PRINCIPAL

AND GOAL FIDELITY

AND VERIFIABLE FACTS

AND MATERIAL DISCLOSURE

AND VISIBLE INTERESTS

AND FREE CHOICE

AND VALID AUTHORITY

AND CONSISTENT REPRESENTATION

AND WITHDRAWAL

AND GENUINE OPPORTUNITY TO CHALLENGE

Work done without passing these gates may optimise behaviour. It does not qualify as GBO that protects human benefit.

The chapter's conclusion

Behavioural optimisation is powerful.

It can influence:

• which information an agent reads,
• which options it considers,
• whom it selects,
• when it asks a question,
• which action it takes.

That power can be used in two different ways.

The first:

Clarifies the facts. Connects evidence. Establishes suitability. Limits authority. Protects people's right to choose and challenge.

The second:

Inflates signals. Conceals material facts. Shifts the user's purpose. Suppresses alternatives. Expands old consent. Makes cancellation and challenge difficult.

The first is GBO. The second is behavioural manipulation. The difference is not merely the technology being used.

It is whose will is being protected.

GBO is not intended to imprint a brand on an agent's mind, increase a platform's transaction count or make a system an invisible decision-maker in place of a person.

Its purpose is for the agent to act in ways that remain grounded in reality, faithful to the user's purpose and aware of authority boundaries; assess options honestly; and protect people's right to withdraw.

GBO's most fundamental ethical conclusion is therefore this: behaviour optimised by reducing a person's freedom to make an informed choice is no longer qualified behaviour. For that conclusion to have real force, however, putting people at the centre in theory is not enough.

A person must be able to:

• stop the agent,
• withdraw authority,
• correct its memory,
• challenge a choice,
• cancel an automated operation,
• reach an accountable human.

Otherwise, “human oversight” is only a slogan.

In the final chapter, we will connect this structure to a basic principle of human control that I have defended throughout the book:

The right to stop the machine

However intelligent, fast or useful a system may be, it must not ignore a valid request to stop or withdraw authority. If stopping itself could cause harm, it must follow a path to a safe state. One of the basic tests of whether a machine serves people is whether it can stop safely when an authorised person asks it to.