8 · Verification: Segment-by-Segment Acceptance and Evidence

8.1 · Acceptance framework

We verify the three scientific properties (scientific, fair, real-time) against acceptance criteria stated before the evidence. The accessibility property is verified against the fidelity-boundary criteria of the presentation layer (Section 6.8). The construct-validity property is verified against the acceptance criteria of Section 8.6. Evidence is labeled by strength: where the window is short, evidence is illustrative, not statistical.

8.2 · Scientific acceptance

Criterion: the invariants in Section 6.9 hold on the published data, and the scale behaves as a slow-anchor / live-signal split. Evidence. The conservation identity was confirmed on a settlement tick (Proposition 6.9); the audit inequality (Proposition 6.14) holds per-tick over the observation window. The separation of time scales is visible in the semantics of the two components: FA is a monotone stock, RI a decaying flow; over an observation window as short as the current one the anchor is expected to move little while the live signal moves much more. A quantitative confirmation of this time-scale split requires a longer window and is listed as future work (Section 8.9). The FA snapshot that supports this discussion is given in Appendix G with its collection date; it is a snapshot, not a time series, by design (the live signal is not reproduced here). Counterfactual that would fail the test. A violation of the conservation identity on a settlement tick, or a violation of the audit inequality on any tick, would fail the criterion. Neither occurred over the window. Because the window is short, the time-scale-separation evidence is illustrative, and the claims are invariant checks and design properties, not hypothesis tests (Amrhein et al., 2019).

8.3 · Fairness acceptance

Criterion: reconciliation is possible, source coverage is disclosed, corrections are logged, and the measured inputs are third-party-led. Evidence. The dual-track audit (the inequality of Proposition 6.14) was used as an automatic reconciliation check between the absolute-heat and signed lines across the window. Source coverage and the missing-value protocol are documented (Appendix B); a correction log records adjustments to the ledger. The organization and tool attributions in the first scope (Section 3.6, Appendix C) were checked item-by-item against primary sources at writing time to the extent web verification was available. Counterfactual that would fail the test. A missing-value handling that could not be reconciled against its registered classification, or a ledger correction with no log entry, would fail the criterion. The audit trail is designed so that the failure would be detectable in principle.

8.4 · Real-time acceptance

Criterion: the three-layer latency calibers (collection / pricing / display) are measured, and the response to a known event is demonstrable. Evidence. The pipeline is run at the calibers of Section 7.3: collection on the order of 1–5 minutes, pricing at L2 bucketing, display at minute level. Because the window is short and the live signal is not reproduced here, the event-response demonstration is illustrative rather than a statistical test: the settlement-tick behavior (frozen anchor, batch injection, centering) and the reopen semantics were observed to match the design. A full latency and event-response study is future work.

8.5 · Accessibility acceptance (index layer)

Criterion: the presentation-layer artifacts (a number, a line, a standard) are produced by a well-defined mapping that preserves ranking and relative magnitude, and pass the “does not violate scientific fairness” check. Evidence. The mapping is the projection of Section 6.8; by construction it preserves the ranking (it is order-preserving on the flagship line) and declares its aggregations. The V1 snapshot in Appendix G serves as the “understand at a glance” demonstration of the output layer only. This is evidence about the presentation layer's fidelity, not scientific evidence about the world.

8.6 · Construct validity acceptance

The construct-validity claims of Section 4 need their own acceptance framework, because they are claims about a theoretical construct rather than about the system's internal consistency. Table 5 states, for each validity claim, the acceptance criterion, the evidence status, and the failure condition.

Table 5. Construct-validity acceptance criteria. “Evidence status” uses now (evidence present at writing), planned (design defined, data not yet collected), and running (continuous monitoring).

The framework is the paper's answer to “how will anyone know the index measures influence?” — not by the system's assertion, but by a running program of tests whose failure conditions are stated in advance. The distinction between now, planned, and running is the honesty of the framework: at writing, the system has internal-consistency evidence and preliminary discriminant evidence, and is missing the external convergent evidence that only time and independent data can supply.

8.7 · Quantitative FA sensitivity design

Composite indicators are only as trustworthy as their sensitivity analysis; the methodological literature is unambiguous that an indicator whose ranking flips under small weight perturbations is not a measurement, it is a choice (Saisana et al., 2005; Nardo et al., 2005). The system therefore adopts a quantitative sensitivity design before the V1 release is treated as authoritative. The design follows the uncertainty and sensitivity framework of Saisana et al. (Saisana et al., 2005): perturb inputs and weights over stated ranges, and report how much the rankings and the invariants move.

Table 6. Quantitative FA sensitivity design. Each factor is a free parameter in the planned experiment; “perturbation” states the design range.

The two reportables are (i) the rank-flip matrix (how many pairwise rankings reverse under each perturbation, summarized by a flip count and the largest single-factor flip) and (ii) the invariant stability (conservation and audit inequalities under perturbation, which must hold by construction and are re-checked numerically). The design's acceptance criterion is stated in advance: a factor whose removal or re-weighting flips more than a stated fraction of the cross-sectional rankings is flagged as a dominant factor, and the committee decides whether the factor's dominance is a fact about the field or a defect of the composition (Section 10). The sensitivity analysis also covers the Leamer-style extreme-bounds question (Leamer, 1983): report the range of each unit's rank as weights range over a plausible box, so that a reader sees not one ranking but the robustness region of the ranking.

8.8 · Failure modes and threat matrix

Manipulation resistance (Section 7.5) and data integrity (Section 9.1) are formal enough to deserve a consolidated threat matrix. Table 7 lists the failure modes the design defends against, the mechanism of the failure, the defense, and the residual risk that remains after the defense. The residual risks are the honest remainder; they are the input to the governance and monitoring agenda (Section 10).

Table 7. Failure-mode and threat matrix. “Defense” is the design mechanism; “residual” is what remains even with the defense in place.

Twelve rows is not a complete threat catalog; it is the catalog the design is engineered against, and it is published so that the residual risks are visible to the same scrutiny as the defenses. The governance agenda of Section 10 is organized around the residual column: every residual risk has an owner (an audit check, a monitoring loop, or a governance review).

8.9 · Limitations

Because FA is a slow-moving anchor by design (Section 5.3), the first point-in-time snapshot is a meaningful reading on its own; claims about how the anchor moves between successive settlements await a longer window and are deferred. The board is small (twelve Originators in the first assessment), so relative measures are coarser than they will be after expansion. Overlapping capability sources create correlated mid-board noise; ties are reported transparently. The unified calibration experiment — a batch re-estimation of the V1 reference parameters against held-out evaluation — is future work (Section 12.2); the released values are V1 production values, not a placeholder awaiting calibration. RI is strongly correlated with events and times, and its numerical realization cannot be reproduced by an external party by design; this is a boundary of the measurement (its live component is not end-to-end verifiable) rather than a hidden defect, and it is the price of manipulation resistance (Section 7.5). Two further boundaries are stated for completeness. First, the external convergent evidence for construct validity is planned, not yet collected; the index is a measurement system whose validity case is in progress (Section 8.6). Second, the composition's numerical weights are V1 values whose robustness region is not yet measured; the sensitivity design of Section 8.7 is the plan, and the weights are production values, not a calibrated optimum.

Back to contents  |  ← 7 · Trustworthiness: Why the Index Can Be Scrutinized  |  9 · Data Dictionary, Pipeline, and Audit →