Emergent
A capability appearing abruptly above a specific model scale rather than improving smoothly.
What it is
During pretraining, as model parameters and compute increase past certain thresholds, certain downstream tasks suddenly become solvable. For instance, multi-digit arithmetic or complex logic problems might show near-zero accuracy across models with under ten billion parameters, but jump significantly for models exceeding seventy billion parameters. This is not a smooth logarithmic curve, but a phase transition where a new latent ability abruptly clicks into place. The exact mechanism remains an active area of research, but it correlates with the network crossing a structural threshold where internal representations can finally model the underlying rules.
Why it matters
You need to know this so you do not waste engineering time trying to prompt engineer smaller models into performing complex reasoning tasks they fundamentally cannot handle. If your feature requires multi-step logic or complex JSON extraction, writing better prompts will not bridge the gap if your underlying model is too small. Conversely, upgrading to a larger model instantly unlocks capabilities that previously required brittle orchestration or complex programmatic workarounds.
In practice
In production, this manifests as a sudden drop in error rates or a completely new category of user queries succeeding when you swap your API backend to a larger model tier. You do not configure a specific knob for emergence, but you must select an appropriate model tier during architecture planning based on the complexity of the tasks your prompts demand.
The tradeoff
The main misconception is that small models can eventually match large ones on complex tasks if given enough prompt engineering or compute at inference time, whereas emergence proves that absolute scale is sometimes an immovable prerequisite.