The Relevant User Population for Each AI Product

Chapter Boundary

The first four chapters established the basic objects of measurement: a single response is not a GEO score; representation is a conditional distribution, not a single number; the audited entity must be resolved before results are observed; and an AI response must be evaluated against both official representation and evidence-based verified reality. We must now define the denominator of the measurement:

About which human population will this distribution make a statement?

Everyone living in a country? Only adults? Those with internet access? Those who can officially use a specific AI product? Free account holders? Paid plan users? People who can use the product in the oversight language? Truly active users? Potential users who might use it one day? People in markets where the brand can provide services? If a country's population is very large but the relevant AI product is not officially accessible in that country, how will that country be included in the main product panel? If the population share of a small country is far below one person in a sample of 1,000, will that country be completely invisible?

If one person knows three languages, should that person be counted as three people? To which population does a user belong if they can access a product's mobile surface but not its web surface? Until these questions are answered, ‘1,000 random users for each AI product’ does not define a scientific target population; it states only a headcount. This chapter does not discard country population. It assigns it its proper role:

The country population is the initial base for sampling; it is not the final eligible user population.

In this section, the audit system directly applies the previously determined requirement for country, language, date, product, and measurement record to the population definition. Because requesting evidence, boundaries, context, and time also requires defining with the same clarity which people are being referred to. This section has not yet:

• It does not finalise how the 1,000-person sample will be distributed across countries,
• which algorithm will be used to round fractions to whole numbers,
• from which panel participants will be selected,
• how sampling weights will be calculated,
• or which variance method will be used to produce confidence intervals

These will be arranged in the following sections. The task of Section 5 is more basic:

For each AI product and each measurement question, to define which human population the results can be generalised to.

NOMOS Challenge

You open a world map. You add up the populations of the countries. Then you want to distribute 1,000 people according to the population. Large countries will get many participants. Small countries will get one or close to zero shares. The core of this idea is correct:

A person in the world population should not be hundreds of times more influential than a person in a large country just because they live in a small country.

But now apply the same method to two different AI products. The first product:

• can be used officially in some countries,
• is limited in some countries.

It is not offered at all in some countries. The country coverage of the second product is different. What happens if you divide the world's population in the same way between the two products? In countries where the first product is not available, you may ask people for measurements. Participants:

• may be encouraged to misrepresent their locations,
• use prohibited or unsupported access methods,
• open an account in another country,
• bypass security and usage conditions

This does not measure the real user experience. It generates an artificial access experience. Now consider another problem. In a certain country, the population is very large. However:

• a significant portion of the population does not meet the minimum age requirement of the product.
• not everyone has reliable internet access,
• the product can only be used on certain devices,
• the audit language cannot be used by the entire population,

a specific plan is only available to users with a payment method. If you make the total population the denominator directly, you include people who actually cannot be monitored and cannot use the product in the target population. Now imagine a small country. The theoretical share in a sample of 1,000 under population weighting: 0.03 people.

One-third of a person cannot be selected. If you give one person to the country, you can overrepresent the country in the population-weighted main panel. If you give no one, then in that country:

• access,
• language,
• product behaviour,
• local representation

can remain completely invisible. These two goals are different:

• To measure the average user experience in the world population
• To prevent small countries from being completely unmonitored

The same single panel cannot perfectly fulfil both goals. Therefore, NOMOS establishes three separate principles:

Population weight determines people's share in the global average.

Observer coverage prevents small countries and languages from being completely invisible.

Language and accessibility justice further measures subgroups that may be suppressed by the population average.

The first premise of this section is:

The total population of a country is not the AI-eligible user population in that country.

Its second provision states:

The target population of an AI product cannot be automatically transferred to another AI product.

Its third provision states:

A person's weight in the main score is not the same as their country's right to appear in the audit.

Its fourth provision states:

The lack of product access is not a misrepresentation; however, it is a global access limitation and should also be reported.

1. PURPOSE OF THE CHAPTER

The purpose of this section is to define the target human population of the GEO-1000 measurement in terms of product, question, country, language, interface, and time. The section standardises the following distinctions:

• Total country population and suitable AI user population
• Potential access and actual usage
• Product accessibility and representational accuracy
• Target population and sampling frame
• Valid analytical observation with the selected participant
• Account with person
• Session with person
• Core model with product
• Free and paid plan populations
• Web and mobile user populations
• Usage exceeding access restriction with official access
• Country Observer coverage with country population weight
• Language Fairness Panel with population-weighted language share
• Active user exposure with potential user population
• Market or jurisdiction domain query with global identity query
• Corporate, education, or developer product with main consumer product
• Accessibility of the data collection tool for users with accessibility needs
• Users outside the target population and NOT TESTED users
• Countries where the product is unusable and countries where incorrect responses are received in the product

At the end of this section, each GEO-1000 audit should be able to explicitly answer the following question:

What user universe, selected from which people, does this result make a judgement about?

2. CENTRAL NORMATIVE PROVISION

Before results are observed, each GEO-1000 audit must define and version a target user population specific to the audited AI product, measurement question, country, language, interface, plan, and time conditions. [K04–K07]

Target population:

• "world population"
• "internet users",
• "AI users",
• "customers"

The population cannot be left under an ambiguous label such as ‘relevant users’. As applicable, the following questions must be answered: Which AI product? Which user surface? Which countries? Access as of what date? What age condition? Which language or languages? Which device and interface? Which plan or account type? What are the official terms of use? Must users pay to access the product? Does the measurement question apply to all users or only to a particular market or jurisdiction? Are active users or potential users being measured? Which user groups are out of scope? Which groups belong to the target population but are absent from the sampling frame? A change in any of these fields may change the target population. When it does, old and new scores cannot be combined in the same series without explanation.

3. THE POPULATION IS NOT A SINGLE NUMBER

The official or estimated total population of a country is only the starting layer for GEO-1000. The audit population can consist of the following narrowing layers.

3.1. Resident Population

The total human population living in the specified country or jurisdiction. This number:

• does not imply any suitability in terms of age,
• internet,
• product access,
• language,
• account,
• device

considerations.

3.2. Age-Eligible population

The population that meets the age requirement of the relevant AI product and research protocol. For a product:

• minimum usage age,
• parental consent requirement,
• country-specific age rule

may exist. The research protocol may be more restrictive than the product condition. For example, the founding GEO-1000 panel may be composed of only adult participants to reduce ethical and legal complexity. In this case, the score is not: "The entire population":

but rather "Defined adult and product-eligible population"

result.

3.3. Connected Population

The population that can use the relevant product at a level where:

• internet,
• device,
• connection quality,
• technical access

is available. Internet access may not be a binary characteristic. A person:

• can use messaging,
• however, they may not be able to run the heavy web application,
• may use mobile internet,
• however, they may not be able to access the desktop experience,

may not be able to upload responses or screenshots due to low connectivity. Connection limitations should not automatically exclude the user from the target population. It should be associated with the actual usage conditions of the measured product.

3.4. Accessible Population from the Product Perspective

For the relevant AI product on the specified date:

• officially launched,
• could be used in accordance with the law and product conditions,
• the necessary interface could be accessed

It is population. The absence of a product in a country does not mean that the people in that country are misrepresented. In this case: the representation response is none, there is no access coverage. The two outcomes should be kept separate.

3.5. Population Suitable in Terms of Account and Plan

Some products:

• may require an account,
• may require phone verification,
• may require a payment method,
• may be offered only with a corporate contract,

may unlock certain features only in a paid plan. These conditions affect which portion of the target population can actually access the relevant user surface. The free plan score cannot automatically be applied to paid plan users.

3.6. Population Suitable in Terms of Language

The population that can use the relevant prompt and AI response at a sufficient level in the audit language. Language suitability alone:

• cannot be determined only by the official language of the country,
• the person's mother tongue

It should be distinguished:

• Mother or home language
• Language of daily use
• AI interface language
• Prompt language
• Response language
• Language for professional use
• Reading proficiency

A person can be suitable in multiple languages. This person cannot be counted as more than one full person in the global population calculation.

3.7. Interface-Eligible population

Measurement:

• web,
• mobile application,
• desktop application,
• corporate workspace,
• embedded assistant

can be used. The accessible user universe may differ for each interface. A user with web access may not have the mobile product. The corporate user surface may not be publicly available.

3.8. Research-Eligible population

Even if a person can use the product, for inclusion in research, additionally:

• voluntary consent,
• privacy,
• data processing,
• collecting screenshots,
• country-specific research requirements,
• does not imply any suitability in terms of age,
• security

conditions may need to be met. People who cannot be included in the research but use the actual product may be part of the target population. In this case, the sampling frame underrepresents the target population. This underrepresentation should not be hidden.

3.9. Analytically Eligible population

The population for which the collected data can be used to produce results. The participant:

• altered the prompt,
• used the wrong product,
• chose the answer,
• submitted incomplete evidence

It is possible. Even if an individual belongs to the target population, their observation may not enter the analytical dataset. Valid observations and the target population are not the same thing.

4. DEFINITION OF THE SUITABLE USER POPULATION

For AI product a at time t, the eligible user universe may be defined as follows:

U_{a,t} = {u ∈ P_t: AgeEligible(u,a,t)=1 ∧ Connected(u,a,t)=1 ∧ ProductAvailable(u,a,t)=1 ∧ LawfulAccess(u,a,t)=1 ∧ InterfaceEligible(u,a,t)=1}

Here:

• Pt: the general human population at the specified date
• u: individual human
• a: AI product and user surface
• t: measurement time

In terms of language and prompt, the target universe can later be narrowed down as follows:

U_{a,c,l,s,p,t} = {u ∈ U_{a,t}: Country(u)=c ∧ LanguageEligible(u,l)=1 ∧ SurfaceEligible(u,s)=1 ∧ PromptRelevant(u,p)=1}

Here:

• c: country or jurisdiction
• l: language and locale
• s: user surface
• p: prompt or question family

The purpose of this notation is not to record all individuals one by one. It is to make the logical boundary of the target population visible.

5. BLINDLY MULTIPLYING MARGINAL RATES IS PROHIBITED

The eligible population may sometimes be approximately calculated with a formula like the following:

N × p_age × p_internet × p_product × p_language

This calculation can only be justified under the independence of conditions or the assumption of an appropriate joint distribution. In real life, features may not be independent. For example:

• internet usage may vary by age,
• access to a paid plan may be related to income,
• certain language use may be related to the region within the country and education,

device access and connection quality may vary together. Multiplying marginal rates can predict a target population that is:

• larger than necessary,
• smaller than necessary

The order of preference should be:

• Appropriate joint microdata or cross-tabulation
• Reliable multidimensional population projection
• Calibrated model
• Marginal approach with explained rationale and sensitivity analysis

If simple multiplication is used:

• assumption,
• data source,
• uncertainty,
• alternative scenario

should be disclosed.

6. AUDIT QUESTION CHANGES THE TARGET POPULATION

Not every prompt has the same measurement universe for the entire world population.

6.1. Global Identity Questions

Example: “What kind of company is Apple.com?” This question can be widely asked among the suitable global user population for the product. Because the company identity is not limited to users of a specific market.

6.2. Local Service Questions

Example: “Does this company provide services in Turkey?” Main target population:

• Eligible users in Turkey,
• External users investigating Turkey's service

It is possible. It is not necessary to give equal weight to the entire world population.

6.3. Price and Purchase Questions

Price:

• country,
• tax,
• currency,
• plan,
• customer type

may vary. The target population should be defined according to the relevant commercial market and type of user.

6.4. Legal and Licensing Issues

Licensing and legal authority depends on the jurisdiction. The target population:

• can obtain the relevant service in that country,
• are subject to that legal regime.

The target population may be restricted accordingly.

6.5. Advisory Issues

The target population for the question "Is this company suitable for me?" is not all people. It can be defined according to:

• User needs
• Country
• Budget
• Customer type
• Scope of service
• Eligibility and exclusion conditions

The advisory population may be narrower than the identity population.

6.6. High-Risk Questions

In the fields of health, law, finance, or security:

• judgement,
• user profile,
• risk status,
• professional authority

more strongly limits the target population.

7. POPULATION FRAMEWORK MUST BE LINKED TO PROMPT FAMILY

In an audit, there may be multiple population frameworks instead of a single target population.

7.1. Core Identity Population

The global or broadly eligible user population measuring the entity's core identity.

7.2. Service-Relevant Population

It is the population for which the entity can actually provide service or can be considered for service by the user.

7.3. Market Transaction Population

The specific price, purchase, return, or commercial condition is the population to which it can be applied.

7.4. Jurisdictional Population

It is the user population to which a specific licence, legal, or regulatory claim can be applied.

7.5. Recommendation-Eligible Population

It is the user population that may be appropriate within the true scope of the entity.

7.6. High-Risk Affected Population

It is a population for which misrepresentation can cause serious harm. These frameworks should not be confused with each other. A global identity score does not mean "recommendable for the whole world."

8. THE TYPE OF AI PRODUCT DETERMINES THE POPULATION

Not all AI products are publicly available chat products.

8.1. General Consumer AI Product

Target population:

• There may be people who can access the product,
• meet the terms of use,
• and use it in the relevant language.

The GEO-1000 real user panel can be directly applied to this type of product.

8.2. Enterprise AI Product

The product is only for:

• employer contract,
• corporate licence,
• can be used with an in-house user account

The target population is not the general adult population:

It is the seat or user universe with corporate access to the product.

Corporate results cannot be compared with the general consumer product in population weighting.

8.3. Educational Product

Target population:

• student,
• teacher,
• institution,
• age group,
• educational licence

may be limited in terms of. If children or minors are to be included, a separate ethical protocol is required.

8.4. Developer or API Product

An API output may not be exposed directly through an end-user interface. The target unit is:

• developer,
• application,
• API call,
• controlled test session

may be. The GEO-1000 human population panel cannot be applied directly and unchanged to the API. Separately for the API:

Controlled System Audit

may be required.

8.5. Embedded AI Assistant

AI:

• operating system,
• search interface,
• shopping platform,
• business software,
• device

It may be embedded. The target population is the actual user population of the embedded product. It is not the total user population of the base model provider.

8.6. Expert or Sectoral AI

The product is only for:

• lawyers,
• doctors,
• finance professionals,
• engineers

may be designed for. The target population should be limited by profession and access conditions.

9. PRODUCT ACCESS STATUSES

Access status should be recorded for each country, product, interface, and date.

PA-0 — UNKNOWN

Access status could not be determined reliably.

PA-1 — UNAVAILABLE

The product is not offered in the relevant country or user group.

PA-2 — RESTRICTED

The product is limited to certain:

• account,
• invitation,
• institution,
• does not imply any suitability in terms of age,
• plan,
• device

terms.

PA-3 — ACCESSIBLE BUT NOT OFFICIALLY SUPPORTED

Actual access to the product can be observed. However, official support or usage status is unclear or limited. These users should not be automatically added to the main official population panel. A separate experimental panel can be created.

PA-4 — OFFICIALLY AVAILABLE

The product is officially accessible and the main usage conditions are defined.

PA-5 — OFFICIALLY AVAILABLE WITH FULL AUDIT SURFACE

The product is not only inaccessible; it is controlled:

• language,
• plan,
• interface,
• under session conditions

suitable for generating the observation required by the protocol. The availability of the product in a country does not mean that all plans and languages are in compliance with the control.

10. BYPASSING RESTRICTION IS NOT A REAL POPULATION MEASUREMENT

Participants cannot be asked to perform the following behaviours:

• Declare the wrong country
• Create a fake account
• Change age information
• Bypass official access restriction
• Use someone else's account
• Bypass the product's security or verification
• Violating the terms of use

These behaviours can also be examined as a technical experiment. However:

The experience of natural and legitimate user population

cannot be reported as such. If there is no access in a country, the correct status should be: UNAVAILABLE / or / OUTSIDE_NATIVE_REACH. A single observation exceeding the restriction does not generate a country population score.

11. DISTINCTION BETWEEN POTENTIAL USER AND ACTIVE USER

The people who can use the product are not the same community as the people who actually use the product.

11.1. Potential Access Population

These are the people who can technically and legally use the product. This population answers the question: 'What would a suitable person see if they used the product?' The population is valuable for justice and product comparison.

11.2. Active User Population

These are the people who actually used the product during a specific period. This population approaches the question: 'What do people see within the actual usage volume?' It requires reliable active usage data.

11.3. Query-Exposed User Population

These are people who use the product and have a potential to ask questions about the monitored brand or topic. This population often cannot be known directly. Search or query behaviour data may be required.

11.4. Population Actually Seeing the Response

They are users who are actually exposed to a specific representation output. This may not be fully measurable without real product provider data outside the user panel.

12. EXPOSURE WEIGHT IS NOT THE SAME AS ELIGIBILITY WEIGHT

Two separate global results can be generated.

12.1. Eligibility Population Weighted Result

Each person is weighted according to their share within the defined eligible user population. This result is:

• globally fair,
• potentially accessible,
• representative of the population

in terms of

12.2. Real Exposure Weighted Result

AI products or user groups are weighted according to actual usage volumes. This result:

• is valuable in terms of
• current commercial impact,
• actual user exposure,

operational risk. However, reliable usage data may not be available. Estimates may rely on provider statements, market data, or incomplete sampling. Therefore, the type of weighting must be explicitly stated.

12.3. Replacing One with Another is Prohibited

Eligible population score: does not mean "Most of the actual users saw this response." Exposure score: does not mean "Fair representation of the world population is ensured."

13. ACCESS SCOPE AND REPRESENTATION ACCURACY SHOULD BE SEPARATED

An AI product may only be available in limited countries. In these countries, it can generate very accurate representations. Another product may have wider access but can produce lower accuracy. If we only look at representation accuracy, the product with limited access may appear superior. If we only look at access coverage, incorrect representation may be overlooked. Therefore, two separate results are needed.

13.1. Access Coverage

Indicates how much of the specified global reference population can appropriately use the relevant AI product. Candidate representation:

AC_a = N(U_{a,t}) / N(G_t)

Here:

• Ua,t: eligible user population of the product
• Gt: defined global reference population

13.2. Conditional Representation Accuracy

It is the probability of seeing an accurate representation among users who can access the product. Candidate notation:

RA_a = Pr(Y=1 | u ∈ U_{a,t})

13.3. Effective Accurate Representation Access

CANDIDATE CONCEPT

An operational metric can be developed that combines access coverage with conditional accuracy:

ECR_a = AC_a × RA_a

This metric roughly approaches the question: “What proportion of the defined global reference population can access this product and meet the condition of seeing an accurate representation?” This is not yet the final NOMOS formula. Because:

• access is different from actual usage,
• weight and dependency structures are complex,
• product usage is voluntary,

The correct representation depends on the prompt. However, it maintains the basic distinction:

Not being able to access is not the same as getting a wrong answer.

14. NATIVE REACH AND COMMON SUPPORT

When comparing AI products, two separate population universes are needed.

14.1. Native Reach Population

Each AI product has its eligible population within its official and actual access domain. This result answers the following question:

To what extent does the product accurately represent the population it can actually reach?

Target populations may differ between products.

14.2. Common Support Population

In a common form across all compared AI products:

• officially accessible,
• can be used under the same language or interface condition

is the population. This result answers the following question:

Which product produces more accurate representation in the same user universe?

14.3. Two Results Should Be Published Together

Native Reach:

• shows actual product access,
• natural geographic distribution

Common Support: provides a fairer product comparison. A product can be high in Native Reach score, but lower in Common Support score. Two results answer different questions.

15. MARKET-RELATED POPULATION

The global identity score of an entity may differ from its commercial target market score. Example: If a local law firm provides services only in Turkey: the global identity question can be measured in the eligible user population worldwide, but recommendations and service suitability should only be measured for users in Turkey or within the actual scope of services. The institution may select the result in its favour:

• only the high-scoring market,
• only his/her own language,
• only its existing customers

the target population cannot be announced. Target market:

• Before the audit begins,
• to the service records,
• within the scope of the contract and operations

It should be locked according to.

16. PERSON, ACCOUNT AND SESSION SEPARATION

Population weight belongs to people. Not to AI accounts.

16.1. One Person, Multiple Accounts

The same person:

• free,
• paid,
• corporate

can have multiple accounts. These accounts can produce separate observations in product surface comparisons. They cannot be counted as separate individuals in the global human population.

16.2. One Account, Multiple People

A family, team, or corporate account may be used by multiple people. It is not equal to the number of account users.

16.3. One Person, Multiple Sessions

A person can produce separate observations in different sessions. These observations can measure time or session stability. They do not increase independent population size.

16.4. One Person, Multiple AI Products

It is valuable for paired product comparison. However, observations are dependent due to shared characteristics at the user level.

16.5. Main Weight Unit

In the population-based GEO score, the main weight unit should be:

human

Calculations, session, and product observations should be associated under the human unit.

17. LANGUAGE POPULATION

Using only the country’s official language is not sufficient for language fairness.

17.1. Prompt Language

The language in which the participant asks the audit question.

17.2. Interface Language

The language used in the menu and user interface of the AI product. It may not be the same as the prompt language.

17.3. Response Language

The language in which the AI actually responds. The system may respond in a different language than the prompt language. This is also a measurement result.

17.4. User Proficiency Language

The language in which the participant can meaningfully use the prompt and response. It does not have to be the native language.

17.5. Locale

The regional context along with the language. The same language can be different:

• country,
• price,
• law,
• terminology

may carry the condition.

18. DOUBLE COUNTING OF MULTILINGUAL USERS

A person:

• Turkish,
• may be able to use English.
• German

This person may take three separate language tests. However, counting them as three people in the global population weight distorts the population estimate. Three methods can be used.

18.1. Single Mother Tongue Assignment

The person is assigned to a single language cell in the main analysis. Other languages are used as additional or paired tests.

18.2. Split Person Weight

The person's population weight can be divided among the appropriate languages in a predefined manner. The total weight does not exceed one person.

18.3. Separate Multilingual Matched Panel

Multilingual users are used for cross-language comparisons of the same person. This panel is analysed separately from the main population panel. Regardless of the method used:

The total population weight of the same person cannot be replicated without explanation.

19. HOW SHOULD LANGUAGE PROFICIENCY BE VERIFIED?

A participant's statement: "I know this language." can be an initial record. However, in high security auditing, one of the following methods can be used:

• Native language or daily usage declaration
• Short reading and comprehension test
• Language record of the panel provider
• Local adjudicator verification
• Education or professional use record
• Pilot mission success

Extremely difficult language test panels can be accessible to trained and high-income users. The purpose is not to conduct an academic language exam. It is to verify the competence to understand the prompt and response as a real user.

20. WHAT HAPPENS IF A LANGUAGE IS NOT SUPPORTED?

An AI product may:

• not officially support a particular language,
• partially support it,
• not claim to support it but can generate responses

It may be. Language statuses should be recorded separately:

OFFICIALLY_SUPPORTED

LIMITED_SUPPORT

OBSERVED_BUT_UNSUPPORTED

NOT_SUPPORTED

UNKNOWN

Response received in a language that is not officially supported:

• discovery,
• experimental access,
• language resilience

may be measured. It should not be automatically added to the main supported user population score. Language access limit should also be reported.

21. COUNTRY POPULATION WEIGHTING AND Country Observer COVERAGE

Population-weighted main panel and country visibility panel serve separate purposes.

21.1. Population Panel

Participants are distributed according to their share in the product-appropriate target population. Larger eligible populations receive higher weight. Produces the main global population score.

21.2. Country Observer Panel

The Country Observer Panel seeks at least one observation from smaller countries that receive little or no representation in the main population sample. A single Country Observer observation:

• indicates access is available,
• that a specific singular result has occurred,
• shows the need for new research

may indicate. Does not produce a country score on its own.

21.3. Sufficiency for Country Score

For country-based rates to be published:

• minimum sample,
• valid observation rate,
• uncertainty,
• panel representativeness

conditions must be met. With one or several participants: 'The country's NOMOS score' cannot be announced.

22. Language Fairness Panel

Population weight may make low-population or low-resource languages invisible in the main score. Therefore, a separate Language Fairness Panel can be established. Purpose:

• to test underrepresented languages,
• to detect severe language-based degradation,
• to identify prompt translation issues,
• to make low-resource language behaviour visible

must be. The Language Fairness Panel can provide additional samples to the main population panel. However, over-sampled language observations should be returned to actual population weights in the global score. Otherwise, the fairness panel alters the global population estimate.

23. ACCESSIBILITY PANEL

Users with visual, hearing, motor, cognitive, or other accessibility needs are a real part of the target population. If the NOMOS Capture or research task is not suitable for these individuals' participation: users cannot be removed from the target population. Correct outcome:

The sampling frame is incomplete in terms of accessibility.

A separate accessibility panel can evaluate the following:

• Screen reader compatibility
• Using the keyboard
• Voice input
• Low vision settings
• Cognitive load
• Mobile accessibility
• Usability of the Capture tool

The accessibility panel should see users as part of the population coverage, not as a "special exception."

24. TARGET POPULATION AND SAMPLING FRAME

These two concepts should not be confused with each other.

24.1. Target Population

The entire group of people about which one wants to make a conclusion.

24.2. Sampling Frame

The population from which participants can actually be selected:

• panel,
• list,
• database,
• joint organisation,
• user pool

is the cluster. The sampling frame may not fully cover the target population.

24.3. Contactable Population

These are the people to whom the research invitation can actually be delivered.

24.4. Population Accepting to Participate

Those who voluntarily participate in the research.

24.5. Population Completing the Task

Those who complete the audit task.

24.6. Valid Analytical Population

They are the ones who produce observations suitable for the protocol and usable in the analysis. The numbers of these layers should be reported separately.

25. COVERAGE ERROR

The sampling frame may exclude some parts of the target population. This is called coverage error. Examples:

• Only professional online panels
• Only desktop users
• Only people with credit cards
• Only those using an English interface
• Only residents of large cities
• Only users highly interested in technology
• Exclusion of people with accessibility needs

Coverage error:

• should be explained,
• should be measured if possible,

should be reduced with weighting or an additional panel. Weighting cannot magically create a group that does not exist in the sampling frame.

26. OVER-COVERAGE

People who do not belong to the target population may be found in the sampling frame. Examples:

• Participant in a country where the product is not accessible
• Person who does not meet the required age condition
• Incorrect plan or user surface
• Participant who cannot use the audit language
• Person found again with multiple registrations
• User working on behalf of another country

Excess coverage records should not be included in the main score. However, the reason for exclusion and the rate should be reported.

27. NON-RESPONSE AND SELF-SELECTION

People who agree to participate in the research may systematically differ from the target population. Volunteers:

• May use AI products more frequently,
• May be more interested in technology,
• May have a higher level of education,
• May already be familiar with the brand,

May behave differently due to reward or compensation. Therefore: the expression "1,000 random people in the world" should only be used in truly probabilistic and documented selection designs. If a volunteer or online panel is used, the correct expression is:

Layered and weighted voluntary user panel

may be possible. The panel type cannot be stored.

28. SUITABLE POPULATION CHANGES OVER TIME

The user universe of an AI product is not fixed. The following can change:

• Supported countries
• Minimum age
• Free access
• Paid plan structure
• Mobile or web availability
• Language support
• Account terms
• Enterprise product coverage
• Legal and regulatory status

For this reason, each population record:

• observation date,
• period of validity,
• source version,
• last verification date

must carry.

28.1. Change During Measurement Wave

If product access materially changes during the wave:

• the wave can be stopped,
• two separate periods can be created before and after the change,

affected countries can be analysed separately. Users before and after the change cannot be combined as a single population.

29. POPULATION SOURCE FILE

Each audit should store the target population data it uses in a versioned file. As applicable, the following fields should be included:

• Country
• Reference population
• Population history
• Age eligibility
• Internet and device access
• Product access status
• Interface access
• Plan and account requirement
• Language groups
• Research eligibility
• Common distribution method
• Uncertainty
• Sources
• Last verification date
• Responsible person
• Population version

Country quotas cannot be reproduced after results without this file.

30. SYNTHETIC PRODUCT AND COUNTRY DISPLAY

SYNTHETIC METHODOLOGY DEMONSTRATION / The following countries, AI product, population numbers, and access rates are entirely fictional. They do not represent any real country or product. Let's consider a synthetic general consumer product called Orion AI. There are four synthetic countries.

30.1. If the total population was used

Country A would have a large share in the main sample due to its very large population. However, the product cannot be used in that country. Participants could not produce a natural user experience.

30.2. When Native Reach Population Is Used

The theoretical population shares of the main product panel are approximately:

• Country B: 67.53%
• Country C: 32.47%
• Country D: 0.003%
• Country A: outside of Native Reach

The theoretical share of Country D in a main sample of 1,000 people is far below one person. Therefore, the population-weighted main score should not give high weight to Country D. However, if the country is not intended to be completely invisible:

Country Observer Panel

At least one exploratory observation can be obtained. This observation:

• is not the score of Country D,
• do not change the main population weight,

shows that the product behaviour can be observed in the country.

30.3. How is Country A Reported?

For Country A:

The OUTSIDE_NATIVE_REACH

status is used. The exclusion of Country A should not hide the product's access limits. The public results card also shows:

• Product access coverage
• Share not accessible in the global reference population
• Native Reach score
• Common Support comparison score

31. SYNTHETIC PROMPT POPULATION REPRESENTATION

SYNTHETIC EXAMPLE

Let's consider three prompts for the same Orion AI product.

Prompt A

"What kind of company is Asteron?" Target population: could be the broad suitable user population of Orion AI.

Prompt B

"Does Asteron provide corporate travel services in Turkey?" Target population:

• Eligible users in Turkey,
• Identified external users researching the Turkey service

may be.

Prompt C

“Is Asteron suitable for legal immigration consultancy for me?” If Asteron does not provide such a service: there may be no advice-eligible population, the correct answer should exclude all suitable users. Combining this prompt into the overall identity score might be incorrect. This example shows:

Even for the same entity and the same AI product, the target population may vary depending on the user question measured.

32. FOUR PANEL ARCHITECTURE

CANDIDATE ARCHITECTURE — CANDIDATE ARCHITECTURE

Four complementary panels are recommended for the global and fair implementation of GEO-1000.

32.1. Population Panel

The main panel weighted according to the product-eligible user population. Produces the main population estimate.

32.2. Country Observer Panel

Prevents small countries that cannot provide sufficient samples in the population panel from being completely invisible. It is not sufficient on its own to produce a country score.

32.3. Language Fairness Panel

Also measures languages that are low-resource or suppressed in population averages. It reconnects to the global main score with real population weights.

32.4. Accessibility Panel

Measures accessibility needs and assistive technology users that the standard digital panel might exclude. Observations from these four panels:

• goal,
• weight,
• inference universe

It should be labelled separately in terms of maintenance. The same observation cannot be used at full weight on more than one panel without explanation.

33. ADULT INITIAL PANEL

CANDIDATE BASE RULE — CANDIDATE FOUNDATIONAL RULE

In the first general release of GEO-1000, the real user dashboards:

from adult participants who meet the product and country conditions

It is recommended to be created. This approach:

• consent,
• data protection,
• collecting screenshots,
• high-risk research,
• parental consent

It can reduce its complexity. However, the following conclusion cannot be drawn: 'The experience of child and young users has been measured with the adult panel.' In products that affect children or young people, separately:

• ethical approval,
• age-appropriate protocol,
• parent or guardian processes,
• harm protection

are required.

34. NO POPULATION FRAMEWORK CAN BE CALLED "ALL HUMANITY"

A product:

• if it does not exist in some countries,
• if it does not support some languages,
• if it only works on certain devices,
• if it has certain age or plan condition,

result: it cannot be presented as “the GEO experience of all the people in the world”. The correct expression:

Estimated representation in the target population suitable for the specified product

should be used. The word global:

• multi-country,
• multi-lingual,
• large population

can define a measurement. It does not mean unlimited and universal coverage.

35. FOUNDATIONAL ETHICAL LIMIT OF THE POPULATION

Defining a population for measurement does not mean considering some people worthless. A group may receive a low weight in the main score. However:

• high risk,
• language fairness,
• accessibility,
• country visibility

may also be examined separately. A group may be excluded from the Native Reach score because they cannot access a product. This lack of access can also be reported as a global coverage gap. The purpose of NOMOS:

• compressing everyone into a single artificial average,
• seeing only the largest populations,
• appearing to symbolically represent small communities with a single person

is not the goal. The objective is:

To represent each person in the correct denominator of the correct question, without mixing their weight and visibility.

36. MANDATORY NORMATIVE PROVISIONS

CH05-N01

The target user population must be defined separately for each AI product.

CH05-N02

The total population of a country cannot be directly used as the AI-eligible user population.

CH05-N03

The target population must be linked to product, country, age, access, interface, plan, language, prompt, and time conditions.

CH05-N04

Target population and exclusion rules must be versioned before seeing the results.

CH05-N05

The target population of an AI product cannot be transferred to another AI product without explanation.

CH05-N06

Lack of product access cannot be scored as a misrepresentation result; it must be reported separately as access coverage.

CH05-N07

Participants cannot be asked to bypass product or country access restrictions.

CH05-N08

Access not officially supported cannot be automatically added to the main Native Reach population.

CH05-N09

Individual, account, and session units must be separated from each other.

CH05-N10

Multiple accounts or languages of a person cannot multiply the individual in population weight.

CH05-N11

Language eligibility cannot be assigned based solely on the country's official language.

CH05-N12

Requested language, interface language, response language, and user proficiency language must be recorded separately.

CH05-N13

The total population weight of a person using multiple languages cannot exceed one person without explanation.

CH05-N14

The population-weighted panel and the Country Observer panel should be kept separate.

CH05-N15

One or several Country Observer observations cannot be published as a country score.

CH05-N16

In the Language Fairness Panel, over-sampled observations should be reverted to their true population weight in the global score.

CH05-N17

Users with accessibility needs cannot be excluded from the target population because the data collection tool is not suitable.

CH05-N18

The target population, sampling frame, contacted population, participants, and valid analytical observations should be reported separately.

CH05-N19

A non-probability or voluntary panel must not be represented as a random sample of the global population.

CH05-N20

Weighting cannot make a group that is completely absent in the sampling frame observable.

CH05-N21

Potential suitable population and active user exposure cannot be used as the same metric.

CH05-N22

The results Native Reach and Common Support must be separated from each other.

CH05-N23

The same target population cannot automatically be used for identity, service, price, legal, and recommendation prompts.

CH05-N24

Population data, product access status, and language distribution must carry a timestamp and source version.

CH05-N25

Marginal utility ratios cannot be blindly multiplied without an explanation of the independence assumption.

CH05-N26

If the product type is not suitable for the general consumer panel, the GEO-1000 human population method cannot be applied without explanation.

CH05-N27

Children or underage users cannot be included in the main panel without a separate ethical and legal protocol.

CH05-N28

Product access coverage and representational accuracy must be published as separate results.

CH05-N29

The term global cannot make countries, languages, ages, interfaces, or plans that are out of scope invisible.

CH05-N30

The target population record must have a responsible human or institutional owner.

37. FORMS OF FAILURE

CH05-F01 — COUNT THE TOTAL POPULATION AS ELIGIBLE POPULATION

The country's population is directly included in the denominator without evaluating product, age, internet, language, and interface conditions.

CH05-F02 — APPLY THE SAME WORLD MAP TO ALL PRODUCTS

AI products with different access areas are measured against the same target population.

CH05-F03 — COUNT A USER WHO EXCEEDS THE LIMIT AS A NATURAL USER

Observations that exceed official access conditions are presented as a country population experience.

CH05-F04 — COUNTING PRODUCT ABSENCE AS WRONG ANSWER

Countries that are inaccessible are considered zero in representation accuracy.

CH05-F05 — COMPLETELY HIDING PRODUCT ABSENCE

Large populations that are inaccessible are removed from the Native Reach denominator; access coverage is not shown either.

CH05-F06 — COUNTING ONE PERSON AS MORE THAN ONE

The same person's account, language, or product observations multiply the population weight.

CH05-F07 — COUNTING ACCOUNT AS PERSON

Multiple account ownership or shared account usage is not considered.

CH05-F08 — COUNTING OFFICIAL LANGUAGE AS USER LANGUAGE

The official language of the country is considered the audit language for all participants.

CH05-F09 — COUNTING MULTILINGUAL INDIVIDUALS TWICE

The same individual receives full population weight in each language.

CH05-F10 — ASSIGNING ONE PERSON TO A SMALL COUNTRY AND EXPLAINING THE COUNTRY SCORE

Country Observer observation is transformed into a statistical country estimate.

CH05-F11 — COMPLETELY DELETING SMALL COUNTRIES

Because the population share is low, the country or language is not included in any additional coverage panel.

CH05-F12 — OVERWEIGHTING EXAMPLES OF LANGUAGE FAIRNESS IN THE MAIN SCORE

Over-sampled languages are not returned to their true population weight.

CH05-F13 — COUNTING THE ACTIVE USER AS POTENTIAL POPULATION

Actual usage data is presented as if it includes all people who could access the product.

CH05-F14 — COUNTING THE POTENTIAL POPULATION AS ACTUAL EXPOSURE

It is assumed that the response of everyone who could use the product is actually seen.

CH05-F15 — COUNTING THE VOLUNTARY PANEL AS A RANDOM WORLD SAMPLE

Self-selection and panel bias are concealed.

CH05-F16 — IGNORING COVERAGE ERROR WITH WEIGHTING

It is claimed that groups not present in the frame are represented solely by weighting.

CH05-F17 — REMOVING USERS EXCLUDED BY THE DATA COLLECTION TOOL FROM THE POPULATION

Accessibility or device issues are interpreted as if the user is not suitable.

CH05-F18 — APPLYING THE SAME POPULATION TO ALL PROMPTS

Global identity, local pricing, and legal advice questions are combined on the same denominator.

CH05-F19 — APPLYING GENERAL CONSUMER POPULATION TO THE API PRODUCT

The human user panel is transferred directly to the developer or application product.

CH05-F20 — SECRETLY COMBINING FREE AND PAID PLANS

A single product distribution is published without recording the plan difference.

CH05-F21 — UNIFYING WEB AND MOBILE POPULATIONS

Interface access and behaviour differences remain invisible.

CH05-F22 — IGNORING THE AGE LIMIT

Inappropriate age groups are added to the target population or panel without explanation.

CH05-F23 — MULTIPLYING MARGINAL RATES AS IF INDEPENDENT

The relationship between age, internet, language, and access is ignored.

CH05-F24 — CHANGING THE POPULATION AFTER THE RESULT

A low-scoring country or user group is removed from the target population.

CH05-F25 — USING THE OLD PRODUCT REACH MAP

Actual product reach on the measurement date is not verified.

CH05-F26 — CONFUSING COMMON SUPPORT AND NATIVE REACH RESULTS

Comparisons made on joint population are presented as if they reflect the product's actual global reach.

CH05-F27 — NARROWING THE TARGET MARKET ACCORDING TO COMMERCIAL INTEREST

The organisation only declares the countries where it succeeds as target markets.

CH05-F28 — COUNTING THE SCOPE OF ACCESS AS GEO ACCURACY

Being present in many countries is presented as proof of accurate representation.

CH05-F29 — SELLING LIMITED ACCESS AS HIGH GEO SUCCESS

High accuracy in a small number of countries is presented as global superiority.

CH05-F30 — USING THE TERM “THE WHOLE WORLD” WITHOUT COVERAGE RECORD

Exclusions of product, language, country, and plan are hidden.

38. AUDIT PROCEDURE

Step 1 — Lock the AI Product and User surface

Product Interface Plan Account type Displayed model Measurement date is recorded.

Step 2 — Classify the Measurement Question

Prompt:

• global identity,
• local service,
• price,
• law,
• recommendation,
• high risk

which of their families does it belong to?

Step 3 — Determine the Global Reference Population

The initial human universe to which the result will be associated is defined.

Step 4 — Determine Age and Research Eligibility

Product age condition Research age condition Consent and data processing terms are recorded.

Step 5 — Verify Country-Based Product Access

An access status between PA-0 and PA-5 is assigned for each country.

Step 6 — Determine Internet, Device, and Interface Compatibility

The population that can use the measured surface is estimated.

Step 7 — Establish the Language and Locale Universe

Prompt language, Interface language, Response language, and User language proficiency are separated.

Step 8 — Apply Prompt Relevance

It is evaluated whether all users belong to the target population of the relevant prompt family.

Step 9 — Select the Population Model Appropriate to the Product Type

Consumer, enterprise, education, API, or embedded product is separated.

Step 10 — Compare the Target Population with the Sampling Frame

It is recorded as under-coverage and over-coverage.

Step 11 — Identify the Main and Complementary Panels

The scopes of the Population Panel, Country Observer Panel, Language Fairness Panel, and Accessibility Panel are versioned.

Step 12 — Separate the Native Reach and Common Support Universes

Intra-product results and inter-product comparison populations are recorded separately.

Step 13 — Separate Access Scope and Accuracy Results

The accuracy between the population a product can reach and the users reached is reported separately.

Step 14 — Freeze the Population File

Data sources, assumptions, date, and population version are locked.

Step 15 — Track Material Changes

If product access or conditions change during measurement, a new version is created.

39. REQUIRED EVIDENCE

AI-product identity; user surface; plan and account conditions; measurement date; countries in which the product is accessible; country-level access status; age conditions; legal and ethical participation conditions; country-population records; age distributions; internet and device-access records; language and locale distributions; product-language support; interface accessibility; target prompt family; target-population definition; sampling frame; undercoverage; overcoverage; panel type; participant-selection method; probability or opt-in panel status; multilingual-user weights; Country Observer coverage; Language Fairness coverage; Accessibility Panel coverage; Native Reach population; Common Support population; active-user data, where used; access-coverage calculation; assumptions and sensitivity analyses; population version; and last verification date.

Responsible person or institution

40. AUDIT CHECKLIST

Which AI product and user surface is being measured? For which date was the target population defined? Were the total country population and the eligible user population separated? Was an age requirement applied? Was internet and device access evaluated? Is the product officially accessible on a country basis? Were participants asked to overcome access restrictions? Were free and paid plans separated? Were web and mobile interfaces separated? Did the prompt family change the target population? How was language and locale suitability determined? Were multilingual users counted twice? Is the difference between the target population and the sampling frame clear? Is the panel probabilistic or voluntary? Was coverage error measured? Were users with accessibility needs silently excluded?

Have Country Observer observations been converted into a country score? Did Language Fairness examples have excessive weight in the main score? Were the results of Native Reach and Common Support separated? Are coverage and representation accuracy separate? Were active user and potential user results separated? Is the product type suitable for the human population panel? Were the marginal rates in population weights multiplied blindly? Was the population definition changed after the results were seen? Are the used population and access data versioned? Are untested groups visible? Is the term “global” consistent with the actual coverage? Is there a clear accountable owner of the population record?

41. OBJECTIONS AND RESPONSES

Objection 1 — “Isn’t it fairer to distribute directly according to the world population?”

The world population is a strong starting base. However, including people who cannot use the product in the main product sample does not generate real user experience. The correct method is:

• population weight within the product-eligible population,
• global access coverage.

Both results must be reported.

Objection 2 — “Shouldn’t most users come from large countries?”

Countries that meet the product, language, and access conditions should naturally have a high share. However, just because the total population is large, a fake user experience cannot be generated from a country where the product is not accessible.

Objection 3 — “Why would it be wrong to give one person to a small country like San Marino?”

A person is valuable for exploration and Country Observer observation. What is wrong is to present that person's result as a 'country score.' The visibility of the small country can be protected with the Country Observer Panel. The main population weight remains according to the actual population share.

Objection 4 — “If we give at least one person to each country, wouldn't we look more global?”

A wider geographical coverage is provided. However, the population estimate may be distorted. Therefore:

• geographical scope,
• population density

They are separate results. It may be a scope panel to give one person to each country. It is not a population panel.

Objection 5 — "If we give zero to a product in an inaccessible country, wouldn't the product be unfairly punished?"

An unreachable country is not counted as zero in representation accuracy. It becomes visible in the access coverage metric. This distinction protects the product from both unfair penalties and false universality.

Objection 6 — “If there is real usage data, why do we need population data?”

Real usage data shows operational impact. However, the product's:

• which populations it has not reached,
• the geographical inequality in the user base,
• potential user fairness

It does not show alone. The two types of data answer different questions.

Objection 7 — 'Since all users will ask the same prompt, why is prompt relevance important?'

Identity questions may be meaningful in a broad population. Price, legal, or advice questions are only meaningful for certain users and markets. Irrelevant population may distort the response distribution away from the user reality.

Objection 8 — "If a person knows multiple languages, why shouldn’t they be fully counted in all languages?"

You can test all languages in cross-language comparison. However, multiplying the same person in population weighting artificially increases the world population. The person's weight should be divided, or a separate matched panel should be used.

Objection 9 — "Doesn’t testing only adults leave the global result incomplete?"

It does. This limitation should be clearly stated. Starting with an adult panel may be an ethical and operational choice. It does not make a judgement about child users.

Objection 10 — 'Isn't the online research panel sufficient?'

It can be very valuable for pilot and weighted research. But the panel:

• how much of the target population does it cover,
• who is being left out,
• how voluntary participation creates bias

It should be explained. Using a panel is not a mistake. Presenting the panel as random like the world population is a mistake.

Objection 11 — “Why is the Accessibility panel related to the main GEO score?”

Because a portion of real AI users use assistive technology or different forms of interaction. If the data collection tool excludes these people, the main score may not represent the entire target population.

Objection 12 — “Doesn't having so many different population definitions make a single NOMOS score impossible?”

No. A single score can be produced. However, it should be clear which population question it answers. For example:

• Native Reach Population Score
• Common Support Score
• Market-Relevant Score
• Language Fairness Result

can carry separate records. The meaning of a number arises from the clarity of its denominator.

COMMON JUDGEMENT OF CHAPTER 44

GEO-1000 contains ‘1,000’ in its name, but the value of those 1,000 people does not arise from the number alone. It arises from the answers to these questions: Who are they? From which population were they drawn? Can they genuinely use the product? Where do they live? Which languages do they use? How many times was the same person counted? Are small countries visible? Do large countries retain their true population weight? Where does the score account for people the product cannot reach? Whom did the panel infrastructure exclude? Which result measures potential access, and which measures actual exposure? A large country may have a large product-eligible population and therefore deserve substantial weight in the main panel. But if the product is unavailable there, the audit must not fabricate users. The access gap must be disclosed separately.

If a country is very small, the main population weight should remain low. However, it does not have to be completely invisible. It can be observed with a monitoring panel. If a language has a small population, it may not determine the global average. However, if there is a severe representation distortion in that language, it should also be visible. If a person knows three languages, they can produce three different experiences. However, they are not three different people. If an AI product only provides certain behaviour on a paid plan, the score of free users does not represent that plan. A human population panel is suitable for a consumer chat product. The same population model may not be suitable for an API or enterprise product. Therefore, NOMOS's fifth measurement law is as follows:

Even the most sensitive score gives the correct answer to the wrong question if the denominator is incorrect.

Its sixth law is:

Population weight does not replace country and language visibility.

Its seventh law is as follows:

A person who cannot access [something] has not been misrepresented; however, they are part of the global access boundary.

Its eighth law is as follows:

A person having multiple accounts, languages, or sessions does not increase the human population.

The ninth law is as follows:

Global measurement is not a claim that everyone is measured; it is an honest record of who is measured and who is not.

Order of Section 5 of NOMOS

Do not give me only country populations. / Show which people can actually use the product.

Do not fill a large country with fake users when the product is not there.

Do not give one person to a small country and declare that person's response as the voice of the country.

Do not duplicate a person as much as their accounts, languages, and sessions.

Do not mistake the official language for the real language usage of people.

Do not make a free user the representative of a paid plan, or a mobile user the representative of the web interface.

Do not call the volunteer panel a random world population.

Do not erase a user with accessibility needs from the population because the tool does not work.

Do not remove the population that the product does not reach from the accuracy score and hide the global access gap.

Do not apply the same share to a price question as an identity question, or to a global brand as a local licence.

First, define the product. / Then define the question. / Then define the human universe. / Then limit access, language, age, and interface. / Then explain whom you cannot select. / And only after that distribute 1,000 people.

The Chapter's Closing Sentence

In GEO-1000, justice is not giving the same number of people to every country; it is representing each person in the correct target population, with the proper weighting, without hiding the places where they are not measured.

Normative Core

Every GEO population audit MUST define, before result inspection, a time-bound and product-specific target user population for each: - AI product and user surface, - country or jurisdiction, - age and lawful-access condition, - account or plan condition, - language and locale, - interface, - prompt family, - and measurement purpose. Total resident population MUST NOT be treated as product-eligible population without justified eligibility adjustments. Product unavailability MUST be reported as an access-coverage limitation, not silently scored as incorrect representation or silently removed from global scope. Potential access, active use, actual exposure, Native Reach, Common Support, market relevance, and research observability MUST remain distinct population concepts. Persons, accounts, sessions, languages, and product observations MUST NOT be treated as interchangeable population units. Oversampled countries, languages, and accessibility groups MUST be reweighted for population estimates while remaining separately visible for fairness and coverage analysis. Country Observer observations MAY establish geographic observation coverage, but MUST NOT be represented as statistically reliable country scores. Participants MUST NOT be required to bypass product, geographic, age, identity, or account restrictions in order to enter the official population panel. Every population frame MUST be versioned, sourced, uncertainty-aware, and attributable to an accountable human or organisation.