Emergence

Definition

An ability is emergent if it is not present in smaller models but appears in larger models, and cannot be predicted simply by extrapolating the performance of smaller models.

In the Literature

Wei et al. (2022)

• Binary threshold definition: absent below scale, present above
• Documents phase transitions in specific capabilities
• Distinguishes slow emergence (gradual on linear scale) from truly discontinuous transitions

Arora & Goyal (2023)

• Statistical framework: competence improves as error fraction $\theta$ decreases
• Key result: Competence on $k^{\prime }$-tuples of skills emerges at similar scaling as elementary skills
• 10× scaling ≈ 2× increase in composable skill count

Michaud et al. (2024)

• Monogenic samples: Single quantum determines performance → sharp phase transition
• Polygenic samples: Multiple quanta → gradual improvement
• Power law frequency of quanta explains smooth aggregate + sharp specific emergence

Ganguli et al. (2022)

• Smooth general vs abrupt specific capability scaling
• Analogy: daily weather (volatile) vs seasonal averages (predictable)

In This Project

Emergence is central to our framework:

• Our complexity filtration ${\mathcal{S}}_{\le n}$ captures emergence strata
• The emergence boundary ${\lambda }^{\ast }(T,ϵ)$ marks when skills become task-competent
• ontological-expansion models how the skill ontology grows with scale

Related Concepts

• scaling-laws — The power laws underlying emergence
• skills — The units that emerge
• composition — How skill combinations exhibit emergent properties
• ontological-expansion — Emergence as ontology growth

References

• Literature Overview and Meta-Analysis
• Clarifying “Algebra of Skills Ontology”