How should an organisation prove that an AI explanation faithfully reflects the real decision process and was not invented afterwards?

Direct answer

Link the explanation to a record preserved when the decision was made. That record shows the model and system versions, prompts, data used, selected information, policy, tool calls, human interventions and applied outcome. Match every material statement in the explanation to one of these items of evidence. Where appropriate, remove or change a claimed factor and test whether the result changes. If the record does not support the text, label it as a possible post-hoc account rather than a faithful explanation.

In plain language

Imagine a restaurant bill written from memory after the meal. The total may look right, but it does not prove which dishes were actually ordered. A reliable bill matches the records made when the order was placed.

Why this matters

People readily trust an explanation that sounds reasonable. If it is not tied to the real decision, they may be told to correct the wrong data, the actual error may remain hidden, or accountability may be assigned falsely.

Do not confuse

A plausible explanation sounds intelligible; a faithful explanation agrees with the evidence of the real decision.
A post-hoc explanation is produced after the decision and may be accurate or invented.
Provenance asks where a record came from; causality asks which element changed the result.
The existence of an event log does not prove that the log is complete, accurate or unaltered.
Reproduction seeks a similar result; a fidelity test examines the explanation's causal claim.

What should you do?

Preserve time-of-decision copies of the model, data, prompt, policy, tool and human versions.
Create a time-linked trace of inputs, information selection, tool calls, human intervention and the applied outcome.
Protect the integrity of the event record and decision snapshot with hashes, access controls and a change trail.
Match each material claim in the explanation to the supporting decision record.
Where appropriate, test behavioural fidelity through factor removal, counterfactual, repeat and sensitivity tests.
Do not present unsupported or merely possible accounts as faithful explanations; state the assurance level.
When the system changes, revalidate the explanation instead of carrying it forward to the new version.

How do you audit it?

Is the explanation linked to the correct decision identifier and a snapshot from the time of decision?
Can the model, prompt, data, information selection, tools, policy and human interventions be traced?
Does every material reason in the explanation actually appear in the record?
Are the completeness and integrity of event records protected?
Was behavioural fidelity assessed by testing, or only by the quality of the prose?
Was a post-hoc or provider-generated explanation presented as a certain internal cause?
Was an old explanation used after a version change without renewed validation?

Limit

With opaque and complex models, complete internal causality cannot always be proved. A decision trace shows only the observable process and may not expose every internal mechanism. In that case, state the assurance level and unknowns explicitly.