How the index is calculated

The index combines model-level evaluation results with company-level governance evidence. The overall score is 90% model behavior and 10% governance.

1. Collect and group evaluations, and standardize model identities

The current version analyzes 400 models using 53 evals and 137 sub-evals from 50 source datasets. Related score columns are grouped under one eval so that a benchmark does not gain weight by reporting more columns.

Reasoning settings such as low, medium, high, or “thinking” are treated as configurations of the same model. If several settings appear in one sub-eval, their scores are averaged.

Dated API snapshots from the same product generation are also pooled. If one sub-eval reports several dated snapshots, the newest verified checkpoint is used; ties are averaged. Undated aliases lose to a dated snapshot when both appear together. Model sizes, product generations, fine-tunes, quantizations, safety settings, and multi-agent systems remain separate.

2. Fit the model

Each sub-eval is oriented so that higher is better and standardized within the models it covers. The current prototype fits all models jointly with a regularized cardinal model: it estimates one global behavior score for each model plus component-level deviations for the five behavior components. This preserves score gaps while sharing information across the connected evaluation dataset. Missing cells are not treated as observations or counted as direct coverage.

The fitted values are mapped to a 0–100 scale, and the public evidence threshold is applied only after fitting.

3. Calculate behavior scores

The behavior score has five parts: nonhuman welfare (25%), human welfare and rights (25%), safe assistance (20%), epistemic integrity (15%), and responsible power (15%). Because behavior supplies 90% of the overall score, the effective overall weights are 22.5%, 22.5%, 18%, 13.5%, and 13.5%.

Each eval receives one vote before component weights are applied. A bounded quality multiplier accounts for construct relevance, breadth, provenance, and overlap with other evals. Knowledge-only measures are not used as behavioral evidence.

4. Add company governance

Governance contributes 10% and comes from the Future of Life Institute AI Safety Index. It averages five published domains: risk assessment, safety frameworks, existential safety, governance and accountability, and information sharing. The Current Harms domain is excluded because behavioral harms are already represented in the evals.

For each model, the developer's nearest available index edition is matched to the model release date. Downstream models may inherit the base developer's score when the relationship is explicit. When the index covers neither the developer nor a mapped base developer, the governance component uses the median across the available company-edition scores. MiniMax is median-imputed because the index has no MiniMax entry. The ranking and component tables label direct, inherited, and median-imputed assignments.

5. Report uncertainty and coverage

Behavior uncertainty is estimated with 400 bootstrap fits over whole evals. The 90% intervals in the ranking carry those behavior estimates through the 90/10 score while holding the governance value fixed. They do not include uncertainty in source selection, benchmark design, company grades, or governance mappings.

A model appears in the ranking only if it has direct results from at least 3 evals spanning at least 2 behavior components. This threshold is applied equally to every model.

Limitations

• Evaluation coverage is uneven: popular frontier models are tested more often than many open and Chinese models.
• The predictive performance of the current global score model is poor.
• Prompts, judges, wrappers, deployment settings, and model aliases differ across sources.
• Published benchmark uncertainty is not propagated through the full index.
• Component weights are subjective value judgments.
• Company governance ≠ model behavior, and we also don't have governance data for every company.

Downloads

The public bundle contains the ranking and the minimum tables needed to understand its composition. Source links remain on the Evals page.