When should a NOMOS GEO audit be considered genuinely independent…

Direct answer

Record the audit scope, method and decision rules before work begins. Raw evidence must be accessible under suitable controls. Another competent auditor must be able to test the result from the same records. The auditor should not approve their own work alone and must be protected from pressure to change the outcome. Disclose interests such as payment, ownership, family relationships, management ties or success targets. A conflict does not always invalidate an audit, but concealing or failing to manage it undermines trust.

In plain language

A chef scoring their own food may be performing internal control; it is not an independent restaurant review. If another employee gives the score, the name is different, but genuine independence is still absent if the owner pressures them not to score poorly.

Why this matters

If the same decision chain defines the method, makes the correction and declares success, errors may remain invisible. Without evidence and a repeat route, 'independent audit' becomes only a reassuring label.

Do not confuse

Internal control is an organisation monitoring its own process; it is not an independent third-party audit.
A second-party review is an assessment by a customer, partner or contractual counterparty.
An independent third-party audit requires a competent external assessor whose conflicts are managed.
Academic validation examines methods and findings through a research process; it is not automatic certification.
Repeatability concerns the same team using the same method; reproducibility asks whether a sufficiently equipped different team can test the result from recorded evidence.

What should you do?

Record the audit scope, acceptance gates, exceptions and decision owner before testing.
Match auditor competence to the task and add legal, security or statistical specialists where needed.
Require written disclosure of financial, organisational, personal, intellectual and performance-related interests.
Where possible, separate the person who made a correction from the person deciding critical closure.
Provide controlled access to raw answers, Truth Pack, question registry, versions, event records and scoring rationale.
Have another competent auditor rescore selected samples blindly or independently.
Disclose method deviations, inaccessible evidence and unresolved disagreements in the final report.

How do you audit it?

Is the auditor approving their own design or correction alone?
Do competence and specialist expertise match the audited risks?
Were interests disclosed and managed?
Were the scope and decision rules changed after results were seen?
Are raw evidence and scoring rationale open to sufficient repeat review?
Can a second auditor reproduce selected results to a reasonable degree?
Was the label 'independent' or 'verified' used while limitations were hidden?

Limit

In closed third-party systems, the same model state, training data or provider infrastructure may not be fully reproducible. The audit should state the evidence it could access and the reproduction limit; labels such as independence or academic validation do not confer formal accreditation.