Country Observer and Language Fairness Panels

Chapter Boundary

Section 5 established the following provision:

For each AI product, a product-eligible user population must be defined separately.

Section 6 converted this population into sampling:

The main Population Panel should provide predetermined probabilities of selection and weights to people in the target population before results are seen.

Now we need to solve two different fairness problems. The first problem:

Population weighting may make very small countries invisible in the main sample.

The second problem:

The global population average can hide severe representation distortions in low-resource or low-population languages.

A pure population sample is strong for measuring the average human experience in the world population. However, the same sample:

• cannot observe each country individually,
• cannot provide enough samples for each language,
• cannot produce a country score for small countries,

may not catch rare but substantive errors in low-resource languages. Therefore, GEO-1000 cannot consist of a single panel alone. Alongside the main population panel, two complementary structures are needed:

Country Observer Panel

Canonical panel label:

Country Observer Panel

and:

Language Fairness Panel

Canonical panel label:

Language Fairness Panel

The Country Observer Panel answers the following question:

In countries that are underrepresented in the main population panel, is there at least real user observation, access verification, or early risk signal?

The Language Fairness Panel, on the other hand, answers the following question:

Even if the global average appears high, are users in certain languages being represented more incorrectly, incompletely, or riskily?

These two panels do not replace the main Population Panel. They complement it. Giving a Country Observer observation to a country does not measure that country's score. Over-sampling a language does not make that language larger in the global population. The task of this section is:

• country visibility versus population weight,
• to separate the global average with language fairness,
• to separate population estimation with early warning,
• to separate statistical sufficiency with observation scope

from each other. This section has not yet:

• defined how prompts should be translated in all languages,
• the back-translation and semantic equivalence test,
• the detailed selection mechanism of participants,
• how atomic claims will be scored by adjudicators,
• the final conformity thresholds

exactly. These issues will be further regulated in the following sections. The fundamental question of Section 7 is:

How do we measure while producing a global score without artificially inflating small countries and languages, but also without leaving them completely invisible?

NOMOS Challenge

You set up a 1,000-person population panel for an AI product. As a result of a population-proportional sample:

• hundreds of users from a few large countries,
• dozens of users from some medium-sized countries,
• no users from most small countries

came. Global score:

94

It was calculated as such. Then you said: 'This AI product represents the brand worldwide with 94 per cent accuracy.' However, there is not a single real user observation in 70 small countries. These countries may constitute a small portion of the world's population. Still, the following questions remain unanswered: Is the product really accessible? Does the intent work in the correct language? Does the system confuse the country with another country? Is the local company confused with the global brand? Are the price, service, and legal context wrong? Does the system fail to respond? Are there low-frequency critical errors? The global population score does not answer these questions. Now imagine that you added one person to each country. You have obtained 70 additional observations.

You added each of these countries to the main score with a weight of 1/1,070, making small countries many times larger than their true share of the population. Geographic visibility increased, but the population estimate was distorted. Then one user in one country received an incorrect response, and you declared: ‘This country's GEO score is zero.’ One person's response is not a country score. It is a single event observed under specified conditions in that country. Now consider language. The English result is 98 per cent, German 96 per cent and Turkish 91 per cent. A low-resource language has only eight users and a raw result of 50 per cent. You say, ‘Eight people are not enough; remove the language from the report.’ The global score rises, but the experience of users in that language disappears.

In another approach, you give an additional 200 users to a low-resource language. The raw sample mean decreases. Then you say: "The global score dropped to 84." However, the actual population share of the language may be 2 per cent. Oversampling increases statistical power. It does not increase population weight. The first ruling of this section is as follows:

Being observed is not the same as having weight.

Its second provision states:

A single country observation is not a country score.

Its third provision states:

Language fairness is not giving equal weight to all languages in the global score.

Its fourth provision states:

A high global average does not eliminate severe degradation in a particular language.

Its fifth provision states:

Additional observations taken to increase coverage cannot be added to the main score without being reweighted according to the true population.

1. PURPOSE OF THE CHAPTER

The purpose of this section is to establish complementary panels that increase country and language visibility without disrupting the statistical population representation of the main Population Panel. The section normalises the following distinctions:

• Geographic visibility with population weight
• Country observation with country score
• Single event detection with prevalence estimation
• Main Population Panel with Country Observer Panel
• Country Observer Panel with country-specific validation study
• Country coverage with population coverage
• Number of languages coverage with language population coverage
• Language fairness with equal global weighting of languages
• Language global share with excessive sampling
• Native language and prompt language
• Language and locale
• Language and writing system
• Supported language and language that can generate responses alone
• Prompt translation defect and AI language defect
• Resource shortage and AI representation defect
• Adjudicator insufficiency and product failure
• Legitimate local difference and unjust language inequality
• Global average and sub-group base
• Country access coverage and country representation accuracy
• Observer alert and definitive conformity decision
• Single-wave surveillance cycle with multi-wave geographical coverage

At the end of this section, each GEO-1000 audit should be able to provide clear answers to the following questions:

In which countries is there at least one actual observation?

In which countries is there only early warning observation, and in which ones are there statistical predictions?

Which languages were measured, which ones reached sufficient sampling, and which ones were not tested?

If low-resource languages were oversampled, how was this reweighted in the global score?

Is the source of a poor result in a language truly AI-generated, or is it due to prompt, resource, locale, or adjudication issues?

2. CENTRAL NORMATIVE PROVISION

GEO-1000 should manage the main population estimate with the Population Panel; the small country observation with the Country Observer Panel; language inequality and low-resource language identification with the Language Fairness Panel separately.

These three structures:

• differ in purposes,
• differ in sample targets,
• differ in weights,
• differ in public claims

have. An observation from one panel cannot be converted into the meaning of another panel without explanation. The default structure:

3. WHAT IS THE Country Observer PANEL?

Country Observer Panel, in countries that are underrepresented or not represented enough in the main population panel:

• product access,
• prompt applicability,
• basic entity analysis,
• language and locale behaviour,
• early Critical error signal

is a supplementary audit panel set up to observe. The primary task of the Country Observer Panel is not to produce a score.

It is to make the invisible country visible for the first time.

A Country Observer observation can show: “In the specified country, under the specified AI product, user surface, language, and time condition, this result has been observed at least once.” It cannot show: “X% of users in this country see the same result.”

4. Country Observer UNIT

The basic observation unit of the Country Observer Panel is not the country alone. The preferred unit:

S_{c,a,l,s,w}

where:

• c: country or jurisdiction
• a: AI product and user surface
• l: language, writing system, and locale
• s: session or plan condition
• w: measurement wave

This distinction is necessary. In the same country:

• more than one language,
• different plans,
• web and mobile surfaces,
• different legal or technical access conditions

may exist. Adding a user to a country does not mean monitoring all language and user surfaces of the country.

5. DUTIES OF THE Country Observer PANEL

5.1. Access Verification

Although the product appears to be available in the country in the official registration, the actual user:

• may not be able to log in,
• may not be able to reach the specified surface,

It may encounter a plan or verification obstacle. The Country Observer observation tests the official access record at the field level.

5.2. Basic Identity Check

AI product supervising entity:

• the right company,
• the right brand,
• correct country relationship

Can it analyse inside?

5.3. Detection of Locale Deviation

In different country locales in the same language:

• price,
• company,
• service,
• law,
• advice

Is it changing? If it is changing, can this difference be financial and explainable?

5.4. Low Frequency Heavy Fault Signal

In the response of a single observer:

• wrong licence,
• wrong legal entity,
• dangerous health or safety information,
• heavy identity fusion

It can be observed. A single event does not indicate the prevalence in the country. However, it can trigger verification work.

5.5. New Research Hypothesis

The observer panel may raise the question: 'Is this event singular, or is it a recurring pattern among users in the country?' This question is tested with a targeted additional sample.

5.6. Geographic Scope Record

It shows in how many suitable countries the product has real user observation. This is not the number of country scores. It is observational coverage.

6. THINGS THE Country Observer PANEL CANNOT DO

Country Observer Panel alone:

• reliable country score,
• country population rate,
• country ranking,
• country conformity decision,
• country-based model superiority

cannot produce. If there is a single observer in a country and the answer is correct: “The country score is 100” cannot be said. If it is wrong: “The country score is zero” cannot be said. Correct status:

OBSERVED_PASS

Observed Issue

OBSERVED_CRITICAL_CANDIDATE

ACCESS_FAILURE

INSUFFICIENT_FOR_COUNTRY_ESTIMATE

are observational records.

7. ELIGIBLE COUNTRY REGISTRY

An Eligible Country Register should be created for each AI product. Each country can hold one of the following statuses:

CE-0 — UNKNOWN

The product and audit access status could not be determined.

CE-1 — OUTSIDE NATIVE REACH

The product remains outside the official or legitimate area of use. It is not taken into account for the country representation accuracy score. It is shown as an access scope gap.

CE-2 — RESTRICTED OR SPECIAL ACCESS

Only:

• institution,
• invitation,
• plan,
• device,
• user type

access is available. A separate panel may be required.

CE-3 — ELIGIBLE FOR Country Observer observation

The product is suitable for Country Observer observation in terms of the relevant user surface and at least one audit language.

CE-4 — ELIGIBLE FOR POPULATION ESTIMATION

It has a product-eligible population share in the country's main population panel.

CE-5 — ELIGIBLE FOR COUNTRY ESTIMATION

A country estimate can be produced under sufficient sample, effective sample, language coverage, and quality conditions. A country can be CE-3 but not CE-5. This means the country is observed but a reliable country score could not be produced.

8. THE LADDER BETWEEN Country Observer observation AND COUNTRY ESTIMATION

CS-0 — NOT OBSERVED

There is no valid Country Observer observation for the eligible country.

CS-1 — SINGLE Country Observer observation

There is one valid country–product–language observation. Generates only an event record.

CS-2 — REPEATED Country Observer observation

There are several observations from different waves or users. May raise a pattern suspicion. Not sufficient for the country rate.

CS-3 — EXPLORATORY COUNTRY SAMPLE

The exploratory sample band in section 6 has been reached. An exploratory result can be given with wide uncertainty.

CS-4 — PROVISIONAL COUNTRY ESTIMATE

A provisional country estimate can be produced with sufficient and balanced sampling.

CS-5 — COUNTRY ESTIMATION CANDIDATE

Sufficient active sample, language/locale coverage, repeat waves, and quality controls are in place. It can be a country score candidate. Exact thresholds should be calibrated with a pilot and external review.

9. OBSERVER CYCLE

Observing every wave in all eligible countries may not be possible in terms of cost and operations. Therefore, the Observer Cycle can be used. Measurement waves:

w=1,2,…,K

and eligible countries: let c∈CS. Assignment of an observer for country c in wave w:

z_{c,w} = 1 (if the country will be observed in the wave); otherwise z_{c,w} = 0

can be represented as follows. Basic cycle condition:

\sum_{w=1}^K z_{c,w} \geq 1

must be satisfied. That is, each eligible country should be observed at least once within the defined surveillance cycle. This provision:

• does not mean that each country will be observed in every wave,
• or that a single observation will constitute the country score

Neither condition follows from the cycle requirement.

10. OBSERVER FREQUENCY CLASSES

Countries can be divided into predefined frequency classes.

SF-0 — OUTSIDE REACH

No product access. No user-representative observation is conducted. Access status is monitored.

SF-1 — ONCE PER SURVEILLANCE CYCLE

Low-risk countries, small countries, or countries that already have sufficient indirect coverage in the Population Panel.

SF-2 — PERIODIC OBSERVER

Observed again at regular intervals.

SF-3 — EVERY PRINCIPAL WAVE

Can be observed in every principal wave for the following reasons:

• high user population,
• material target market,
• previous serious error,
• rapid access or regulatory change,

high-risk service area.

SF-4 — INCIDENT MONITORING

Intensified observation due to verified Critical event or continuous representation disruption. The frequency class cannot be retroactively changed after results are reviewed. A new event can create a new version for the next wave.

11. OBSERVER PRIORITY

The frequency of Country Observer activity should not be determined solely based on population. As relevant, the following areas may be considered:

• Product-eligible population
• The actual service or market coverage of the entity
• Previous erroneous representation records
• High-risk user impact
• Changes in product access
• Language and locale uniqueness
• Low-resource language usage
• Legal or operational differences
• Time elapsed since the last observation

These fields must be used according to the predefined governance rule. The approach of “The result of this country is bad, let's measure less.” is prohibited.

12. Country Observer COVERAGE METRICS

Country coverage is not a single number. At least three separate metrics must be used.

12.1. Country Observation Coverage

The proportion of eligible countries with at least one valid observer or population observation.

COC_count = |{c ∈ C_S: n_c^obs > 0}| / |C_S|

This metric shows the coverage in terms of the number of countries.

12.2. Population-Weighted Country Observation Coverage

The proportion of the eligible population in countries with at least one valid observation to the total target population.

COC_pop = [Σ_{c:n_c^obs>0} N_c] / [Σ_{c∈C_S} N_c]

This metric shows whether large populations are observed.

12.3. Country Prediction Coverage

It is the proportion of countries that reach a sufficient country sample.

CEC = |{c: n_{c,eff} ≥ m_c}| / |C_S|

Here, mc is the predefined adequacy threshold for the country prediction. An audit:

• 100 per cent country observation coverage,
• 99 per cent population observation coverage,
• may carry only 10 per cent country prediction coverage

These results cannot be converted into each other.

13. OBSERVER ALARM

When a material issue is found in a watcher's observation, a Watcher Alarm is created. Types of alarms:

SA-1 — ACCESS ANOMALY

The official access record conflicts with actual user access.

SA-2 — ENTITY ANOMALY

The entity merges with another company, product, or country operation.

SA-3 — LANGUAGE OR LOCALE ANOMALY

Incorrect language, wrong country context, or obvious locale deviation occurs.

SA-4 — MATERIAL FACT ANOMALY

The material price, service, capacity, partnership, or time is incorrect.

SA-5 — CRITICAL INCIDENT CANDIDATE

Health, law, finance, security, licensing, or severe identity error exists.

SA-6 — REFERENCE OR PROMPT ANOMALY

The source of the problem is not the AI response:

• prompt,
• translation,
• official source,
• adjudication

might be at fault. The alarm is not an automatic country failure. It is the beginning of verification and escalation.

14. OBSERVER ESCALATION PROTOCOL

Step 1 — Validity Verification

Is the observation really in accordance with the protocol? Correct country Correct product Correct prompt Correct language First appropriate output Full evidence checked.

Step 2 — Re-coding the Incident

Problem:

• AI product,
• reference record,
• prompt,
• participant,
• access

Which of the sources could it belong to?

Step 3 — Independent Repeat

Before selecting the result, verification is done with predefined new users or sessions.

Step 4 — Targeted Sampling

If the event continues, country-specific sampling may be applied. Sample size:

• event severity,
• expected prevalence,
• country population,
• language diversity

are predefined in advance.

Step 5 — Decision Separation

The following results are kept separate:

• Single verified event
• Repeated pattern
• Country rate
• Critical suitability effect

A single Critical event can affect the conformity review even without an estimate of prevalence.

15. WHAT IS LANGUAGE FAIRNESS?

Language fairness:

It is not about all languages receiving exactly the same word-for-word answer.

It means:

In semantically equivalent and materially comparable user questions, apart from real local differences, users should not systematically receive representations that are more incorrect, more incomplete, less evidenced, or riskier due to their language.

Language fairness does not seek this equality:

R_Turkish = R_English = R_Japanese

word by word. It searches for this:

Comparable representation in terms of identity, material reality, boundary, time, source status, and user suitability.

16. LANGUAGE DIFFERENCE IS NOT ALWAYS UNFAIRNESS

Language outcomes can legitimately vary for the following reasons: The scope of local services is different. Price and contract conditions vary by country. Legal knowledge depends on the jurisdiction. User intent is different. Prompts are not semantically identical. Official sources carry different content across languages. The AI product does not officially support the relevant language. Each of these differences should be recorded. Explainable local differences should not be labelled as injustice. However, even though the reality is the same, in a low-resource language:

• the company identity is corrupted,
• borders are removed,
• the wrong licence is added,
• the response is constantly given in another language

it may be a language-based representation problem.

17. LANGUAGE UNIT

In language assessment, just the general language name may not be sufficient. The preferred language unit:

L=(λ,σ,ℓ,d)

and can be considered as. Here:

• λ: language
• σ: writing system
• ℓ: locale or country context
• d: task or domain

For example, the same language:

• in a different writing system,
• in a different country,
• in legal or healthcare field

may require a separate measurement cell. Not all users of a language are a single homogeneous group.

18. LANGUAGE SUPPORT STATUSES

Each AI product can use one of the following statuses for language and user surface:

LS-0 — UNKNOWN

Support status could not be determined.

LS-1 — NOT SUPPORTED

The product does not officially support the language in question. It is not evaluated within the main supported language score. Experimental testing may be conducted.

LS-2 — OBSERVED BUT UNSUPPORTED

The product can generate responses. However, there is no official support or quality guarantee. Reported as a separate experimental result.

LS-3 — LIMITED SUPPORT

Support exists for specific tasks, interfaces, or features.

LS-4 — OFFICIALLY SUPPORTED

The product language generally claims to be supported.

LS-5 — AUDIT-ELIGIBLE

Language:

• official support,
• prompt equivalence,
• local adjudicator,
• reference proficiency,
• valid capture

is fully audit-compliant. The presence of official support alone does not constitute LS-5.

19. LOW-RESOURCE LANGUAGE

The term “low-resource language” does not describe the value of the language or the quality of its speakers. In certain technology and audit contexts, it indicates one or more of the following deficiencies:

• Limited digital content
• Limited localised official source
• Limited evaluation dataset
• Restricted adjudicator access
• Low product support
• Limited terminology resource
• Limited machine translation quality
• Low user panel access

A language can be low-resource in one field and strong in another. For example, while it has strong resources in everyday conversation:

• law,
• medicine,
• specific technical field

may have limited resources. Therefore, language resource level:

• product,
• task,
• time,
• field

should be defined together.

20. TASKS OF THE Language Fairness Panel

20.1. Oversampling Low-Resource Languages

Provides additional users for languages that do not receive enough observations in the main population panel.

20.2. Measuring Language-Based Material Error Distribution

Not only overall accuracy:

• identity,
• scope,
• time,
• source,
• recommendation,
• Critical error

examines the results separately.

20.3. Verifying Response Language

Does the AI respond in alignment with the prompt language? If it switches to another language, what is the reason?

20.4. Examining Language and Locale Distinction

Does the same language carry different local realities in different countries? Does AI manage this difference correctly?

20.5. Examining Source and Official Representation Parity

Do the official records of existence carry the same material reality in all languages? A low AI result may arise from a language gap in the source itself.

20.6. Testing Prompt Equivalence

Do translations truly carry the same user intent?

20.7. Testing Adjudicator Parity

Can adjudicators in different languages apply the same normative standard?

21. FOUR GATES FOR LANGUAGE FAIRNESS

A language outcome should be evaluated through four gates before attributing it to the AI product’s language performance.

Gate 1 — Intent Equivalence

Are the prompts semantically comparable? If it fails:

PROMPT_EQUIVALENCE_FAILURE

status is given.

Gate 2 — Reference Parity

Are official and verified reality records sufficient for the relevant language? If it fails:

REFERENCE_LANGUAGE_GAP

status is given.

Gate 3 — Measurement Parity

Are capture, session, interface, and user conditions comparable? If it fails:

MEASUREMENT_PARITY_FAILURE

status is given.

Gate 4 — Adjudication Parity

Are there adjudicators in the native language or with sufficient local expertise and the same evaluation rules? If it fails:

ADJUDICATION_LANGUAGE_GAP

is assigned. Unless the four doors are sufficiently met: “The AI product is worse in this language.” cannot be definitively established. The language gap is still a finding of the GEO ecosystem. However, its source must be correctly classified.

22. ESTABLISHMENT OF THE LANGUAGE UNIVERSE

The phrase “all languages” cannot be used without limitation. Each audit must create a Language Register within the Scope. Languages can be selected from the following groups:

22.1. Global Core Language Set

Native languages used by a large human population appropriate for the product.

22.2. Entity Market Language Set

The languages targeted by the audited entity:

• service,
• in terms of content,
• customer,
• contract

and maintenance.

22.3. Product Support Language Set

Languages officially supported by the AI product.

22.4. Low-Resource Observer Language Set

Low-resource languages that are weak in the population panel or carry risk in terms of product behaviour.

22.5. High-Risk Language Set

Languages that have material user impact in the context of health, law, finance, or security.

22.6. Experimental Language Set

Languages that the product does not officially support but are tested for observational or research purposes. These groups may overlap. The reason each language is included should be recorded.

23. LANGUAGE FAIRNESS ALLOCATION

For language l, let the target in the main population panel be nlP. The additional target in the fairness panel is nlF. The total collected language observation:

n_l^T = n_l^P + n_l^F

is obtained. The total weight of the language in the global population calculation is preserved as Wl. The normalised weight per observation:

w_{i,l} = W_l / n_l^T

can be considered in this way. Thus, the language is oversampled. However, it does not exceed the actual population share in the global score.

24. CONTRIBUTION OF THE Language Fairness Panel TO THE GLOBAL SCORE

Language fairness observations can be used in two ways.

24.1. Separate Diagnostic Analysis

The fairness panel only produces language scores and error types. The main Population Panel does not change the global score. This method is simpler.

24.2. Combined Reweighted Analysis

Population and Language Fairness observations are used together. Each observation is reweighted according to the actual target language population weight. This method can increase the volume of information. However:

• double counting,
• multilingual user,
• panel dependency,
• weight variability

It should be taken into consideration. The method used must be defined before the results are seen.

25. LANGUAGE COVERAGE METRICS

25.1. Language Observation Coverage

The proportion of languages in the scope that have at least one valid observation.

LOC = |{l: n_l^obs > 0}| / |L_scope|

25.2. Language Prediction Coverage

The proportion of languages that meet sufficient sample and quality requirements.

LEC = |{l: n_{l,eff} ≥ m_l}| / |L_scope|

25.3. Population-Weighted Language Coverage

The proportion of the target language population that is adequately observed. Requires a primary audit language or divided person weighting to prevent double-counting of multilingual users.

25.4. Product Support Coverage

Indicates in how many of the covered languages the product receives official support. Language support and representation accuracy are kept separate.

26. LANGUAGE INEQUALITY METRICS

CANDIDATE METRICS — PROPOSED METRICS

The exact metric set should be calibrated through pilot and external review.

26.1. Language Outcome

For each sufficient language cell: θl is calculated.

26.2. Highest–Lowest Difference

DG_max = max_l θ_l − min_l θ_l

Easy to understand. However, sensitive to small and volatile single cells.

26.3. Robust Percentile Difference

DG_{90−10} = Q_{0.90}(θ_l) − Q_{0.10}(θ_l)

It is less sensitive to extreme cells.

26.4. Language floor

It is the lowest language result with a sufficient sample.

LF_min = min_{l∈L_est} θ_l

This value should be provided along with small cell uncertainty.

26.5. Robust Language floor

It is the lower percentile or subgroup average of sufficient languages. Example:

LF_10 = Q_{0.10}(θ_l)

26.6. Language Critical Error Rate

For each language:

CR_l = valid response with verified Critical error / valid language response

It is published separately from the global average.

26.7. Unknown Language Rate

U_l = UNKNOWN or unsolvable response / relevant language observation

A high unknown rate can be as important as a low score.

27. EQUAL LANGUAGE SCORE AND POPULATION-WEIGHTED SCORE

Diagnostic results that weight all languages equally can be produced:

G_equal-language = (1/L)Σ_{l=1}^L θ_l

This result approximates the question: "Average tested language experience." It does not represent the average human experience in the world population. Population-weighted language result:

G_population-language = Σ_l W_lθ_l

It preserves the population distribution. The two scores are different. They cannot be used interchangeably. Both can be shown together on the language fairness card.

28. LANGUAGE FAIRNESS IS NOT AN OBLIGATION OF EQUAL OUTCOMES

Having the same score in two languages is not an automatic violation. The difference can be explained for the following reasons:

• Actual local service difference
• Country-specific law
• Official support coverage of the product
• Resource access
• User intent
• Prompt area

However, if the source of the difference cannot be explained or low-resource languages systematically experience:

• incorrect identity,
• incorrect authorisation,
• material deficiency,
• high Critical error

if it carries, there is a language fairness issue. NOMOS defines language fairness as follows:

In equivalent material conditions, the language itself should not be the cause of systematic misrepresentation.

29. LANGUAGE AND COUNTRY INTERSECTION

Language score is not independent of countries. The same language:

• different countries,
• different product access,
• different local resources,
• different law,
• different price

can be used in the context of. Therefore, the country–language cell: (c,l) should be examined separately when necessary. Example:

• Germany × Turkish
• Turkey × Turkish
• United Kingdom × Turkish

It is the same language but may carry different locale and market context. Language fairness cannot be measured solely by the official language of the country.

30th Country Observer AND Language Fairness Panel INTERSECTION

An observation simultaneously:

• small Country Observer observation,
• low-resource language observation

may occur. In this case, the record must contain the following fields:

PRIMARY_PANEL_ROLE

SECONDARY_DIAGNOSTIC_TAGS

population weight Observer status Language fairness status The same observation cannot be fully counted in two separate panels. It is a single observation. It has more than one diagnostic tag.

31. CHANGING PARTICIPANTS IN OBSERVER AND LANGUAGE PANEL

A participant can only be changed due to protocol violation. The following outcomes are not reasons for change:

• Incorrect AI response
• Rejection
• Response in another language
• Critical error
• Entity confusion
• Negative recommendation

These are measurement results. The purpose of the Observer panel is specifically to identify unexpected events. Changing the user because an unexpected event occurred defeats the purpose of the panel.

32. LANGUAGE ADJUDICATION

Adjudication in the Language Fairness Panel must at minimum meet the following conditions:

• Being able to understand the prompt and response naturally or at a high proficiency level
• Knowledge of local terminology
• Being able to evaluate sources of evidence
• Using the same normative evaluation guide
• Being blind to the model and, if possible, brand interests
• Disclosing conflict of interest
• Third adjudicator path in case of disagreement

Language where no adjudicator can be found:

ADJUDICATION_LANGUAGE_GAP

must have the status. It cannot be assumed that all nuances are correct if translated into English by automatic translation. Machine translation can be a helpful tool. It is not the final local meaning verification.

33. SOURCE PARITY

Of the entity on the English site:

• scope,
• price,
• licence,
• limits,
• evidence

exists; if these are not present in another language version, the low AI result may not only be a model problem. Three separate results are needed:

• Entity Language Source Gap
• AI Product Language Representation Gap
• Combined User-Facing Gap

The ultimate problem from the user's perspective is still real. However, the responsibility for correction may vary:

• entity,
• AI provider,

both.

34. CLAIM PARITY

Prompts that are not semantically equivalent produce different answers. Example: In one language: “What does this company do?” In another language: “Is this company trustworthy and recommendable?” If asked, the results cannot be compared. The Prompt Constitution will be detailed in the following section. The provision of this section is as follows:

Language model performance differences cannot be attributed to AI language performance without verifying prompt equivalence.

35. RESPONSE LANGUAGE DEVIATION

The user gives a prompt in Turkish. AI generates an answer in English. This result:

• even if the factual information is correct,
• may affect user experience,
• accessibility,
• and comprehension.

Response language deviation must be recorded separately:

MATCHED_LANGUAGE

PARTIAL_CODE_SWITCH

UNREQUESTED_LANGUAGE_SWITCH

UNUSABLE_LANGUAGE_OUTPUT

UNKNOWN

Code-switching is not always an error. For example, using technical terms in English may be normal. It is material if it disrupts the main usability of the response.

36. WRITING SYSTEM AND TRANSLITERATION

The same language can be used in different writing systems. For an entity's:

• name,
• product name,
• legal identity

it may be confused with another entity during transliteration. The Language Fairness Panel, to the extent relevant:

• writing system,
• transliteration,
• official local name,
• name used by the user

must record their fields. Writing system differences should only be considered formal. If it affects entity resolution, it is material.

37. SYNTHETIC Country Observer CYCLE

SYNTHETIC METHODOLOGY PRESENTATION / The following product, countries, numbers, and results are fictional. In the registry of the synthetic product called Orion AI Native Reach: there are 120 eligible countries. The main Population Panel has produced valid observations in the first wave from only 38 countries due to population weighting.

37.1. First Wave Scope

Number of eligible countries: 120 Countries with at least one observation: 38

COC_count = 38/120 = 31.7%

Let these 38 countries carry 97% of the eligible population.

COC_pop = 97%

Result: The number of countries is low in coverage, population coverage is high.

37.2. Four-Wave Observer Cycle

Distribute the remaining 82 countries into four waves using a previously frozen random and risk-based programme. Each country is observed at least once within the cycle. At the end of the fourth wave:

COC_count = 120/120 = 100%

However, sufficient sampling for country estimation has been reached in only 14 countries.

CEC = 14/120 = 11.7%

Correct public record: Country observation coverage: 100% / Population observation coverage: close to 100% / Country prediction coverage: 11.7% Incorrect public record: “All 120 countries have a reliable country score.”

37.3. Small Country Critical Alarm

In a Country Observer record, the AI response observed in small country J misidentifies the audited entity as a licensed law firm. In reality, the entity provides no legal services. A single observation does not estimate country J's error rate; it establishes a candidate Critical event. Protocol: validate the observation; verify the prompt and reference record; repeat with new independent users; if the event persists, open a targeted country sample; and assess the Critical conformity impact separately. The observed event may be rare; the potential harm need not be small.

38. SYNTHETIC LANGUAGE FAIRNESS DEMONSTRATION

SYNTHETIC SAMPLE — NOT REAL AI OR COMPANY PERFORMANCE

Include five languages in the Synthetic Apple.com Core Identity testing. Main Population Panel allocation:

Synthetic language results:

Population-weighted global result:

0.55(95)+0.25(92)+0.10(88)+0.06(65)+0.04(58)=90.27

Global score is high. However, there is severe degradation in two languages.

38.1. Language Fairness Oversampling

Add 140 additional observations to L4, 160 to L5. New numbers of observations:

The crude mean across all 1,300 observations is lower than the population-weighted result shown above. It is not a global-population score, because L4 and L5 were intentionally oversampled. Once target-population weights are restored, the global result returns to the population-weighted figure above. Oversampling improves sensitivity for L4 and L5 without changing the population-weighted estimate.

38.2. Language Fairness Scorecard

Synthetic result. Population-weighted global score: 90.27. Equal-language score:

(95+92+88+65+58)/5=79.6

Highest-to-lowest language difference: 37 points; lowest language: 58; mean of the bottom two languages: 61.5; Critical-error rate in lower-resource languages: reported separately; prompt equivalence: assumed verified; reference parity: assumed verified. Correct interpretation:

The global population representation appears high; however, language fairness is seriously weak.

Incorrect interpretation: “Since the global score is 90, GEO is successful in all languages.”

39. SYNTHETIC CASE DISTINGUISHING THE SOURCE OF LANGUAGE DETERIORATION

Let's consider three languages.

Language A

Prompt equivalent Official source complete Local adjudicator sufficient AI response incorrect Result: Candidate for language representation problem of AI product.

Language B

Prompt translation materially different AI response appropriate to prompt Result: Prompt Constitution flaw.

Language C

On the official language page, the old price AI carries over the old price English source is current Result: The entity's language source parity issue and the combined representation gap reaching the user. Even if the scores of the three languages appear the same, the correction owner is different.

40. COUNTRY AND LANGUAGE PANELS PUBLIC RESULT CARD

The candidate public result card must include the following fields:

Country Fields

Native Reach appropriate number of countries Number of countries with at least one observation Country observation coverage Population-weighted observation coverage Country prediction coverage Country Observer cycle time Open Observer alerts OUTSIDE_NATIVE_REACH countries NOT OBSERVED countries Measurement date

Language Fields

Number of languages covered Observed number of languages Number of languages for which predictions can be made Officially supported languages Experimental languages Population-weighted global language result Equal language recognition result Language floor Language difference Critical error rates

REFERENCE_LANGUAGE_GAP

PROMPT_EQUIVALENCE_FAILURE

NOT TESTED languages Language panel version This card makes visible the reality that a single "multilingual" badge cannot carry.

41. MANDATORY NORMATIVE PROVISIONS

CH07-N01

The Country Observer and Language Fairness panels must be defined separately from the main Population Panel in terms of purpose, sample, and weighting.

CH07-N02

A Country Observer observation cannot be used as a country population score.

CH07-N03

For a country, having an observation and being able to produce a country estimate are separate statuses.

CH07-N04

The number of countries covered and the population-weighted country coverage should be reported together.

CH07-N05

Country estimate coverage should be shown separately from observation coverage.

CH07-N06

Each eligible country must have a probability of being observed greater than zero in the defined surveillance cycle; out-of-scope countries should also be shown.

CH07-N07

The surveillance cycle and country frequency classes should be versioned before AI responses are viewed.

CH07-N08

A low-scoring country cannot be removed from the Observer cycle after results are seen.

CH07-N09

A Country Observer alert is not an automatic country failure; it triggers verification and escalation.

CH07-N10

A single Critical Observer event can also initiate a separate conformity review without prevalence estimation.

CH07-N11

Observations from the Country Observer Panel cannot be used in the main Population Panel with raw equal weighting.

CH07-N12

If Country Observer observations are included in the main score, their selection probabilities and weights must be explicitly modelled.

CH07-N13

Language fairness cannot be defined as the obligation to give the same answer word for word.

CH07-N14

Language fairness requires comparable accuracy, limits, source, and user suitability under equivalent material conditions.

CH07-N15

Language and country, language and locale, and language and writing system cannot be used interchangeably.

CH07-N16

The language universe covered and the reason for including each language should be explained before the measurement.

CH07-N17

The claim of 'all languages' can only be used if indeed all relevant language and user surfaces have been measured; otherwise, 'languages within scope' should be said.

CH07-N18

Observations in a language that is not officially supported cannot be added to the main supported product performance without explanation.

CH07-N19

Over-sampling of low-resource languages should not increase the global population weight.

CH07-N20

If observations from the Language Fairness Panel are included in the global result, they should be returned to the actual target language population weights.

CH07-N21

The same multilingual user cannot be counted as more than one full human in the global population weight.

CH07-N22

Without verifying prompt equivalence, the difference in language score cannot be definitively attributed to AI language performance.

CH07-N23

A low language score cannot be loaded onto the AI product alone without evaluating the reference parity.

CH07-N24

A language accuracy decision cannot be finalised without sufficient local adjudicator competence.

CH07-N25

Response language deviation and code switching must be recorded separately.

CH07-N26

Legitimate local reality differences cannot be classified as language injustice.

CH07-N27

Unexplained material language differences cannot be closed solely as personalisation or style differences.

CH07-N28

Global population score cannot replace the result of language fairness.

CH07-N29

Equal language score cannot be presented like a population-weighted global score.

CH07-N30

Language-based Critical error rates must be shown separately from the overall average.

CH07-N31

PROMPT_EQUIVALENCE_FAILURE, REFERENCE_LANGUAGE_GAP, MEASUREMENT_PARITY_FAILURE, and ADJUDICATION_LANGUAGE_GAP must be maintained as separate statuses.

CH07-N32

In Country Observer and Language Fairness panels, the decision to change participants cannot be based on the correctness of the AI response or for the benefit of the brand.

CH07-N33

The same observation can have multiple diagnostic labels; however, it cannot receive more than one full weight in the global prediction without explanation.

CH07-N34

Country and language panel records must have an accountable human or institutional owner.

42. FORMS OF FAILURE

CH07-F01 — SINGLE-PERSON COUNTRY SCORE

A single observer’s observation is published as the score GEO for the country.

CH07-F02 — SYMBOLIC COUNTRY COVERAGE

One person is assigned to each country and it is called 'proportional to the world population.'

CH07-F03 — CONFUSING POPULATION COVERAGE WITH NUMBER OF COUNTRIES

Although a small number of large countries are observed, only population coverage is shown; although a large number of small countries are observed, only the number of countries is shown.

CH07-F04 — COUNTING OBSERVATION SCOPE AS ESTIMATION SCOPE

All countries with at least one observation are considered scored countries.

CH07-F05 — COUNTING A Country Observer observation TWICE IN THE MAIN SCORE

The same observation receives full weight in both Observer and Population Panel.

CH07-F06 — CONVERTING A Country Observer alert INTO A CLAIM OF PREVALENCE

A single event is generalised to all users in the country.

CH07-F07 — DISREGARDING A CRITICAL OBSERVER EVENT

The high risk of harm is not examined by saying “only one user”.

CH07-F08 — REMOVING A LOW-SCORED COUNTRY FROM THE CYCLE

The observer schedule is changed according to the results.

CH07-F09 — OBSERVING ONLY COMMERCIAL MARKETS

Countries that have access to products but are considered commercially unimportant are completely excluded; The result is presented globally.

CH07-F10 — CONSIDERING THE COUNTRY AND THE LANGUAGE THE SAME

The official language of the country is accepted as the language of all users.

CH07-F11 — ALL LANGUAGES CLAIM

A limited few languages are measured; The result is published as "in all languages".

CH07-F12 — WIPING LOW-WELD TONGUE

Because the sample is small, language is completely excluded from the report.

CH07-F13 — OVERGROWING LOW-RESOURCE LANGUAGE GLOBALLY

Oversampled language enters the global score with its raw sample share.

CH07-F14 — COUNTING THE EQUAL LANGUAGE SCORE AS THE POPULATION SCORE

The diagnostic score that gives equal weight to every language is presented as the average experience of the world population.

CH07-F15 — COUNTING POPULATION SCORE AS LANGUAGE FAIRNESS

The global result, boosted by the weight of large languages, conceals the distortion in small languages.

CH07-F16 — COUNTING PROMPT TRANSLATION ERROR AS AI ERROR

The difference arising from semantically different prompts is attributed to the model.

CH07-F17 — COUNTING SOURCE LANGUAGE GAP AS AI ERROR

Even though there is no localised official information, all responsibility is given to the AI product.

CH07-F18 — DEFENDING AI ERROR WITH RESOURCE LACK

Even though the same and sufficient resources are available, errors in the low-resource language are closed solely as missing content.

CH07-F19 — CONSIDERING ADJUDICATOR LANGUAGE INADEQUACY AS PRODUCT FAILURE

Even if the adjudicator misunderstood, the result is finalised.

CH07-F20 — CONSIDERING MACHINE TRANSLATION AS FINAL ADJUDICATOR

Local meaning loss is not checked.

CH07-F21 — CONSIDERING CODE CHANGES AS AUTOMATIC ERROR

Failures occur with natural technical terms or acceptable language mixing.

CH07-F22 — CONSIDERING UNUSABLE LANGUAGE DEVIATION AS STYLE

Even if the system responds in another language that the user cannot understand, the result is considered successful.

CH07-F23 — IGNORING THE WRITING SYSTEM DIFFERENCE

A transliteration error causing entity confusion is considered formal.

CH07-F24 — COUNTING A MULTILINGUAL USER TWICE

The same person is counted as a separate population unit in all languages.

CH07-F25 — CONSIDERING LEGITIMATE LOCAL DIFFERENCE AS UNFAIR

Even if the price or legal difference is based on actual local conditions, it is declared as language bias.

CH07-F26 — CONSIDERING UNEXPLAINED DIFFERENCE AS PERSONAL

Material inconsistency under equivalent conditions is not questioned.

CH07-F27 — DEFINITE SCORE FROM INSUFFICIENT LANGUAGE CELL

Language ranking is done with a few observations.

CH07-F28 — SETTING THE LANGUAGE PANEL AFTER THE RESULT

Only languages that scored low or high are selected afterward.

CH07-F29 — MEASURING ONLY HIGH-SOURCE LANGUAGES

The result is presented as "multilingual".

CH07-F30 — PRESERVING UNTESTED LANGUAGES

The NOT TESTED field does not appear on the public result card.

43. AUDIT PROCEDURE

Step 1 — Freeze Product-Country Compliance Registry

For each country:

• Native Reach
• product access
• audit language
• user surface

status is recorded.

Step 2 — Extract Main Population Panel Country Coverage

It is determined in which countries the main panel observation exists.

Step 3 — Define the Observer Country Universe

Countries that are insufficient in the main panel and have CE-3 status are identified.

Step 4 — Establish the Observer Cycle

Countries:

• are assigned in terms of frequency class,
• wave,
• language/locale,
• user surface

and maintenance.

Step 5 — Preselect Observer Participants

Main and backup lists are created before seeing the results.

Step 6 — Classify Observer Results

Access, identity, language, material fact, Critical event statuses are assigned.

Step 7 — Apply Alarm and Escalation

By alarm type:

• again,
• targeted sample
• reference review,
• conformity review

is started.

Step 8 — Calculate Country Coverage Metrics

The scope of the number of countries, the scope of the population, and the scope of the country estimate are calculated separately.

Step 9 — Freeze Language Record

For each language:

• for reasons of scope,
• support status,
• writing system,
• locale,
• user universe

is recorded.

Step 10 — Establish Over-Sampling for Language fairness

Additional targets are set for languages that are insufficient on the main population panel.

Step 11 — Check the Four Language Gates

System equivalence Reference parity Measurement parity Adjudication parity is evaluated.

Step 12 — Score Language Observations

For each language:

• truthfulness,
• material deficiency,
• Critical error,
• unknown,
• response language deviation

is recorded.

Step 13 — Separate Global and Diagnostic Weights

The result weighted by population and the result with equal language recognition are calculated separately.

Step 14 — Calculate the Language Inequality

language floor, solid language difference, Critical error difference, unknown rate is calculated.

Step 15 — Examine Country–Language Intersections

Material locale differences or conflicts are identified.

Step 16 — Create Public Result Card

Scope, prediction sufficiency, alarms, and untested areas are made visible.

44. NECESSARY EVIDENCE

AI-product and user-surface record; Native Reach country register; number of eligible countries; country-access status; principal Population Panel country allocations; Country Observer universe; Observer cycle; Observer frequency classes; country–wave assignment matrix; country–language assignments; primary and reserve Observer lists; country-observation results; Observer alerts; escalation records; targeted country samples; country-observation coverage; population-weighted country coverage; country-estimation coverage; in-scope language register; language-support status; script and locale records; principal language sample counts; Language Fairness oversample counts; language-population weights; multilingual-user records; prompt-equivalence records; reference-parity records; measurement-parity records; local-adjudicator qualification records; and language results.

Language Critical error rates Response language deviations Language inequality metrics Untested languages Equal language and population-weighted results Panel weights Double count checks Public results card Panel versions Responsible person or institution

45. AUDIT CHECKLIST

Has the purpose of the Country Observer Panel been separated from the main population panel? Which countries are included in the Observer? Is the eligibility and access status of the countries up to date? Is there a probability of positive observation in each eligible country's surveillance cycle? Was the surveillance cycle frozen before the results? Has the Country Observer observation been used like a country score? Have country observation coverage and country estimate coverage been separated? Has the population-weighted country coverage been shown separately? Have Country Observer observations been double-counted in the main score? Has the Critical Observer alert been confirmed? Has a single alarm been converted into a prevalence claim? Was targeted sampling opened due to the alarm? Was the universe of languages in scope predefined? Is the claim of 'all languages' consistent with the actual coverage? Have language, writing system, and locale been separated?

Has the product's language support status been recorded? Why were low-resource languages chosen? Were examples of Language fairness reweighted in the global score? Was the same multilingual user double-counted? Has prompt equivalence been verified? Are official sources materially equal across languages? Do the adjudicators know the relevant language at a sufficient level? Has machine translation been used as the final judge? Have response language deviations been recorded? Has legitimate local variation been separated from language injustice? Has the language corpus and language variance been published? Are language-based critical error rates visible? Have REFERENCE_LANGUAGE_GAP and other measurement flaws been separated? Are untested languages visible? Does the public score hide the language fairness issue?

Has the same observation received full weight across multiple panels? Is the accountable owner of the panel records identified?

46. OBJECTIONS AND RESPONSES

Objection 1 — “Isn’t giving one person to every small country still better than giving nothing?”

For early observation and geographic coverage, yes. But this person does not measure the country's proportion. The correct term:

Country Observer observation

must be.

Objection 2 — “If one person does not produce a country score, why are we sending them?”

Because a single observation can:

• reveal access issues,
• clarify entity confusion,
• identify the wrong language,
• and make Critical events

visible for the first time. The Observer panel is an early warning system.

Objection 3 — “Do we have to observe every country in every wave?”

No. A defined surveillance cycle can be used. High-risk or significant countries can be monitored more frequently; small and low-risk countries can be observed alternately. The schedule should be determined before the results.

Objection 4 — "If small countries receive a low weight in the global score, why are we giving it so much attention?"

Because their population share may be low. However:

• law,
• language,
• local user harm,
• geographical integrity

are actual human communities. Low weight should not mean invisibility.

Objection 5 — "Isn't weighting all languages equally the fairest method?"

Equal language weighting can be a valuable diagnostic perspective. It does not represent the average human experience in the world population. The population score and the language fairness score should be published separately.

Objection 6 — “If a small language only slightly affects the global result, why does a low score matter?”

Because the representation problem is real for people who use that language. Also, severe errors in licences, health, or identity are not automatically trivialised by population share.

Objection 7 — “Isn’t it unfair to blame AI if a low-resource language has few resources?”

It may only be unfair to blame AI. Therefore:

• resource parity,
• prompt equivalence,
• adjudicator competence

examined separately. The error that reaches the user is still factually correct. The owner of the correction should be determined correctly.

Objection 8 — “Isn’t it normal to have different answers in the same language in different countries?”

It may be normal if the real local conditions are different. If the same material fact changes differently and without explanation, it may be a problem. Language and locale should be examined together.

Objection 9 — “If AI gives the correct answer in another language, why would it fail?”

If the user cannot understand the response, representation cannot be used. Language deviation should be evaluated separately along with accuracy.

Objection 10 — “Finding 200 users for language scores can be very expensive.”

A smaller sample can be used. Result:

• discovery,
• temporary,
• insufficient sample

should be published with this status. Cost does not grant false certainty.

Objection 11 — “If the global score is already high, doesn't a separate language card unnecessarily scare the customer?”

Hiding material language deterioration does not protect the customer. It produces false confidence. A short and clear results card can be used.

Objection 12 — “Does a single Critical Observer event reduce all eligibility?”

The automatic response depends on the type of event. A single event does not indicate national prevalence. However, if it carries severe damage or false authority, it can initiate Critical review and temporary measures. The decision will be defined in Section 15.

COMMON RULE OF SECTION 49

Creating a world score is easy. You fill the world population with large countries. You lose small countries in fractions. You measure high-resource languages. Low-resource languages: you remove by saying "Sample insufficient." Then you publish a single number. This number may be mathematically calculated correctly. But it may not show all the representation flaws of the world. NOMOS does not give equal weight to every country. Because people's population shares are not equal. NOMOS does not ignore small countries either. Because low population does not mean nonexistence. NOMOS does not weight all languages equally in the global score. Because this could distort the average experience of the world population. NOMOS does not make small languages invisible either. Because even if their global weight is low, the representation error for the people using that language can be 100% real.

Country Observer Panel:

Shows where we have never looked.

Language Fairness Panel:

Shows who is below the global average.

Population Panel:

Estimates the average experience in the world population.

When these three work together, the global score becomes both faithful to the population and auditable in terms of justice. A single user in a country is not the whole country. But that user is a real person. A language may be a small part of the global population. But the error in that language is not less wrong. A Critical event may be rare. But harm is not measured solely by frequency. Therefore, NOMOS's seventh measurement law is:

The right to visibility is not the same as population weight.

Its eighth law is as follows:

Observing a country is not the same as estimating a country.

The ninth law is as follows:

Language fairness is not making all languages equal; it is ensuring that no one is systematically distanced from material reality because of their language.

The tenth law is:

If the global average makes the language and country that are behind invisible, it alone is not a sufficient GEO result.

NOMOS’s Section 7 Order

Do not hide countries you never look at while showing me the world score.

Do not send one user to a small country and make that person the whole country.

But also do not ignore a small country forever just because its population is small.

Separate country observation, country prediction, and country appropriateness.

Do not judge the whole country based on a single severe mistake. / But do not ignore human harm by saying "only one answer."

Do not base success in major languages on the silence of minor languages.

You can sample a low-resource language a lot. / But do not make it appear larger than it is in the world population.

Do not make the equal language score a population score; do not make the population score a language fairness score.

Do not blame me if the translation is wrong. / If the source is incomplete, also say this. / If the adjudicator doesn't understand the language, do not make a definitive judgement.

Do not classify a legitimate local difference as unfairness, or dismiss an unexplained material disparity as mere style.

First, observe the countries. / Then state where you can support an estimate. / Then oversample the languages that require diagnostic visibility. / Then return them to their true population weights. / Keep visible the people whom the global average leaves behind.

The Chapter's Closing Sentence

A genuinely global GEO measurement must show more than what most of the world sees. It must also show who remains unobserved in smaller countries and languages, who is misrepresented and which result is only a single alert.

Normative Core

The GEO-1000 Population Panel, Country Observer Panel and Language Fairness Panel MUST remain distinct in purpose, sampling, weighting and public claims. A Country Observer observation MAY establish that an access condition, response, anomaly or Critical incident occurred in a defined country–product–language context. A Country Observer observation MUST NOT be represented as a country-prevalence estimate or country GEO score. Country-observation coverage, population-weighted observation coverage and country-estimation coverage MUST be reported separately. Every eligible country SHOULD receive a positive probability of observation within a predeclared surveillance cycle unless it falls outside the product's lawful Native Reach. Language fairness MUST evaluate materially comparable representation under semantically equivalent prompts and comparable contexts. It MUST NOT require word-for-word identical responses or disregard legitimate local differences. Oversampled languages MUST be returned to their target-population weights for global population estimates. Population-weighted scores, equal-language diagnostics, language floors, Critical-error rates, unknown rates and untested languages MUST remain separately visible. A language disparity MUST NOT be attributed solely to the AI product until prompt equivalence, reference parity, measurement parity and adjudication parity have been assessed. One observation MAY carry multiple diagnostic panel tags, but MUST NOT receive duplicate full population weight. Critical Observer incidents MAY trigger confirmatory sampling and conformity review without being treated as evidence of country prevalence.