THREAT CONSTELLATION / CROSS-OBSERVATORY

FILE THR-OBS-004 / OBSERVED / 2026-08-30 / evidence sourced

The Admitted Uncertainty

When an institution becomes more trustworthy by admitting that it cannot meaningfully rank what it is asked to choose between.

制度は、選べないことを認めることで、むしろ信頼できるものになることがある。

A ranking looks precise.

But precision is not the same as knowledge.

When candidates are close enough,

the ranking may say more than the evidence does.

Then the real institutional question becomes:

When should a system stop deciding?

順位は、知識と同じではない。

TRACE THE CONFIDENCE BOUNDARY

Rank · then the ranking loses certainty

Confidence Boundary

Indistinguishable range

Formal definition

The Admitted Uncertainty

A condition in which an institution explicitly recognizes that its evaluation system cannot reliably distinguish between candidates beyond a certain confidence boundary, and changes the selection mechanism accordingly.

評価制度が、ある信頼境界を超えると候補間の差を十分に識別できないことを明示的に認め、その後の選抜方法を変更する状態。

Structural shift

RANK EVERYTHING
ESTIMATE CONFIDENCE
IDENTIFY THE INDISTINGUISHABLE RANGE
STOP PRETENDING TO KNOW
USE AN ALTERNATIVE SELECTION RULE

What should a system do when it can measure a difference but cannot justify treating that difference as meaningful?

UNCERTAINTY

≠ IGNORANCE

The institution still evaluates. It simply stops claiming more precision than the evidence supports.

制度は評価をやめない。証拠が支える以上の精度を主張することをやめる。

Not this

a research-funding explainer
a lottery advocacy page
a critique of peer review only
a generic fairness page
an anti-meritocracy argument

01 · Core thesis

Institutional selection systems often assume they can produce meaningful rankings even when the underlying evidence only supports a threshold-level judgment. Historically this looks like candidates, evaluation, ranking, winner — as if 1st > 2nd > 3rd in a meaningful and defensible sense. Some systems may only be able to establish qualified versus not qualified. After that threshold, differences may be too noisy, subjective, unstable or weakly evidenced to justify precise ranking.

Historical model

CANDIDATES
EVALUATION
RANKING
WINNER

Uncertainty-aware model

EVALUATION
QUALIFICATION THRESHOLD
UNCERTAINTY BAND
ALTERNATIVE ALLOCATION

Possible alternative allocation may include lottery, rotation, portfolio allocation, quota by category, human deliberation, or staged experimentation. Lottery is one observed mechanism. It is not always the correct answer.

lottery
rotation
portfolio allocation
quota by category
human deliberation
staged experimentation

02 · Core model

Traditional

100 PROPOSALS
SCORE
RANK
1 WINNER

Uncertainty-aware

100 PROPOSALS
ASSESS QUALITY
QUALIFIED SET
CONFIDENCE BOUNDARY
ALTERNATIVE SELECTION

The system does not become less rigorous. It becomes more explicit about where rigor ends.

03 · Primary signal / OBSERVED · sourced

When Research Funding Stops Pretending to Know the Winner

SCIENCE / SOCIETY · partial lottery

Institutional selection is beginning to move from maximizing ranking precision toward explicitly representing uncertainty.

Evaluation mechanism: peer review · quality threshold · ranking near a funding line
Confidence boundary: Among proposals judged fundable, fine-grained rank near the funding line often cannot be distinguished with enough confidence to justify a precise winner.
Source fact: Some public research funders have introduced partial or modified lotteries after an initial quality assessment. The Swiss National Science Foundation, after a 2018–2020 pilot, allowed drawing lots as a tie-breaker from 2021, and from 2022 used a Bayesian ranking model to identify proposals too close to separate near the funding line. The British Academy, from 2022, randomizes BA/Leverhulme Small Research Grants among applications that pass a quality threshold — a trial later extended toward 2028. Nature reporting in 2026 described this shift as asking whether decisions are precise enough, not whether science should become random. The Health Research Council of New Zealand has used a quality-threshold lottery for Explorer Grants since the 2010s. These are observed institutional mechanisms, not a proof that lottery is the correct rule in every domain.
Why it matters: The observed move is not from review to chance. It is from pretending that a ranking is exact, to marking where the ranking stops being defensible. Peer review still sets a threshold. Chance, where used, operates among proposals already judged qualified.

Even when selection inside the qualified set is random, the threshold that creates that set remains a powerful, non-random decision.

Observed mechanism

REVIEW
THRESHOLD
LOTTERY AMONG QUALIFIED

Not this

NO REVIEW
RANDOM MONEY

Some research funders have introduced partial or modified lotteries after an initial quality threshold. This is not no review, then random money. Not “all proposals are equal”, but “among sufficiently strong proposals, peer review may not reliably distinguish fine-grained differences.”

Watch whether other ranking-heavy institutions — hiring platforms, admissions, credit, procurement — begin to encode a confidence boundary rather than adding decimal places.

04 · Confidence Boundary

SHIRO & Co. conceptual lens. Not an established universal term.

Confidence Boundary

The point beyond which an evaluation system no longer has sufficient evidence to justify precise ordering.

grant proposals
hiring finalists
university admissions
startup accelerator cohorts
public subsidies
procurement bids
creative competitions

05 · False Precision

SHIRO & Co. conceptual lens.

False Precision

Apparent numerical or ordinal exactness that exceeds the actual certainty of the underlying evaluation.

8.72
vs.
8.68
Rank 14
vs.
Rank 17
63.4%
vs.
61.9%

Does the ranking contain more information than the evidence?

This is especially important in AI-scored systems, where added decimal places can look like added knowledge.

06 · Institutional Humility

SHIRO & Co. conceptual lens. A design property, not a moral posture.

Institutional Humility

The capacity of an institution to encode the limits of its own judgment into the decision process.

EVALUATION CAPABILITY
SELF-ASSESSMENT
LIMIT RECOGNITION
ALTERNATIVE DECISION RULE

07 · Adjacent models

08 · AI ranking

AI enables increasingly granular scoring. Increased score precision does not necessarily mean increased epistemic certainty.

hiring
lending
content ranking
research evaluation
fraud detection
admissions
insurance
procurement

MORE DECIMAL PLACES ≠ MORE KNOWLEDGE

MEASURABLE DIFFERENCE ≠ MEANINGFUL DIFFERENCE

09 · Employment · hypothesis

APPLICATIONS
SCREENING
QUALIFIED BAND
UNCERTAIN DIFFERENCE
ALTERNATIVE SELECTION

If several candidates are genuinely indistinguishable, should rank 1 always beat rank 2?
Could partial lottery reduce false precision in hiring?
Could rotation or trial assignments preserve more information?

These remain observational questions. This page does not advocate employment lotteries as policy.

10 · Education · hypothesis

admissions
scholarships
school placement
competitive programs

Highly selective systems may create enormous consequences from extremely small score differences.

When does selection confidence become smaller than selection consequence?

11 · Markets · hypothesis

Financial systems often produce precise ranking outputs even when input uncertainty is large.

This does not imply that markets should randomize.

credit scoring
investment ranking
venture screening
procurement
insurance underwriting

Potential business layer

confidence-aware decision systems
uncertainty bands
calibrated ranking
alternative allocation rules
counterfactual evaluation

12 · Science · observed

PEER REVIEW
QUALITY THRESHOLD
INDISTINGUISHABLE PROPOSALS
PARTIAL LOTTERY

Partial lotteries may

reduce exaggerated ranking precision
reduce reviewer burden
preserve diversity
acknowledge epistemic limits

They also introduce

legitimacy questions
communication challenges
fairness perception issues

Outcomes are not presented as universally proven. Implementations differ: SNSF uses a near-threshold tie-breaker; the British Academy randomizes among a qualified set; other programmes invert the order with lottery-first designs. The shared observation is the admission of an indistinguishable range, not a single allocation rule.

13 · Randomness as Infrastructure

Randomness as Infrastructure

Not

RANDOMNESS = FAIRNESS

Mechanism when knowledge ends

Randomness can become a deliberate institutional mechanism when deterministic selection would imply more knowledge than the system actually has.

jury selection
Lot-drawing has been used to constitute juries and, in some historical polities, public offices among those already eligible.
sortition
Classical Athenian sortition selected eligible citizens for certain offices by lot, after a qualification boundary.
allocation lotteries
Modern institutions sometimes use lotteries to allocate scarce goods after eligibility is established.
randomized experiments
Random assignment is already treated as infrastructure in experimental design when uncontrolled ranking would confound inference.

Can randomness sometimes protect a system from its own false confidence?

14 · Protected Serendipity

Protected Serendipity

Deliberately preserving allocation capacity for possibilities prediction does not favor.

QUALIFIED
NOT RELIABLY RANKABLE
RANDOM / DIVERSE ALLOCATION
UNEXPECTED OUTCOME BECOMES OBSERVABLE

The Pre-Selected Future proposed Protected Serendipity as a conceptual counterweight. The Admitted Uncertainty provides one possible mechanism. This is a cross-link, not a claim that Protected Serendipity already has its own route.

15 · Counter-hypothesis

lotteries may weaken perceived merit
evaluation may be better than assumed
uncertainty bands may be politically manipulated
institutions may use “uncertainty” to avoid responsibility
randomization may obscure structural inequality
threshold design itself remains powerful

Admitting uncertainty does not remove power. It moves power toward defining the threshold.

16 · Threshold Power

SHIRO & Co. conceptual lens.

Threshold Power

The power exercised through deciding who enters the uncertainty band in the first place.

ALL CANDIDATES
THRESHOLD
QUALIFIED SET
LOTTERY

Even if selection within the qualified set is random, the threshold is not.

Connected · The Pre-Selected Future · Opportunity Gate · The Supervisory Layer

17 · Observation matrix

Only science currently has a sourced implementation on this page. Other rows are hypotheses. No current implementations are fabricated.

Domain	Evaluation	False precision risk	Confidence boundary	Alternative rule	Evidence
SCIENCE	peer review	fine-grained ranking	qualified proposal band	partial lottery	observed
EMPLOYMENT	screening scores	rank 1 vs rank 2 among finalists	qualified candidate band	rotation / trial / other	hypothesis
EDUCATION	exam and file scores	tiny score gaps with large consequences	admissible band	unspecified	hypothesis
MARKETS	credit, investment, underwriting scores	precise rank from uncertain inputs	calibrated uncertainty band	unspecified	hypothesis
PUBLIC ALLOCATION	eligibility scoring	ordered queues beyond a threshold	eligible set	unspecified	hypothesis
CULTURE	jury and prize ranking	forced ordinal taste	shortlist band	unspecified	hypothesis

18 · Evidence levels

OBSERVED

Partial and modified lottery mechanisms exist in research funding after a quality threshold.

EMERGING

Institutions are experimenting with uncertainty-aware selection, including Bayesian ranking of indistinguishable proposals and expanded lottery trials.

HYPOTHESIS

Similar mechanisms could spread to hiring, education, markets or public allocation. No such implementations are asserted here.

19 · Connected Observatories

The Pre-Selected Future
The Pre-Selected Future asks what disappears because a model did not expect it to succeed. The live Observation recording that mechanism is Prediction as Pre-Selection. The Admitted Uncertainty asks what happens when ranking itself is admitted to be unreliable as a basis for allocation.
The Supervisory Layer
The Supervisory Layer asks who verifies, authorizes, ranks or legitimizes growing capability. The Admitted Uncertainty adds: who determines when the supervisor should stop ranking.
Opportunity Gate
Forming
An Opportunity Gate decides whether a possibility receives enough capital, exposure, legitimacy or access to become observable. No verified route exists yet. The Admitted Uncertainty would change the gate from best score wins toward qualified possibility remains eligible.
Employment
Hiring systems often rank candidates after a screen. Whether a qualified band should still produce a strict ordinal winner remains a hypothesis, not an observed institutional rule on this page.
Education
Selective admissions and scholarships can attach large consequences to small score differences. The Education–Employment observatory is the verified host for that domain; no education lottery is asserted here.
Markets
Credit, investment and underwriting systems often emit precise ranks from uncertain inputs. The observation is false precision risk, not a claim that markets should randomize.
Science
Forming
Research-funding bodies are the strongest present domain: peer review, then a quality threshold, then an indistinguishable range. No dedicated Science Observatory route is verified.

20 · Quiet questions

When does a ranking become false precision?
順位は、いつ偽の精度になるのか。
Can a system measure its own confidence?
制度は、自らの確信度を測れるか。
What should happen beyond the confidence boundary?
信頼境界の先で、何が起きるべきか。
Does randomness reduce bias or merely relocate it?
無作為は偏りを減らすのか、それとも移すだけか。
Who defines the qualification threshold?
資格の閾値を、誰が定義するのか。
Can a supervisor recognize when it should not decide?
監督者は、決めるべきでないときを認められるか。
When does selection confidence become smaller than selection consequence?
選抜の確信度は、いつ結果の大きさより小さくなるのか。
Is a transparent lottery more legitimate than an unjustifiably precise ranking?
根拠のない精密な順位より、透明な抽選のほうが正当か。

21 · Final

Precision can make a system look certain.

Trust may require knowing where certainty ends.

When should a system stop deciding?

制度は、どこで「決めない」ことを選ぶべきか。