RLHF
Reinforcement Learning from Human Feedback: training a base model using human preference rankings to optimize outputs for safety, tone, and specific task adherence.
What it is
After initial pretraining on raw text, a model generates multiple completions for a prompt, and human reviewers rank them from best to worst. A secondary reward model learns to predict these human preferences across tens of thousands of comparison pairs. Finally, the primary model is updated using reinforcement learning algorithms to maximize the score from this reward model. This multi-step pipeline turns a raw text predictor into an aligned assistant.
Why it matters
As a backend engineer calling LLM APIs, RLHF determines the out-of-the-box behavior, tone, and refusal patterns of the model you use. If you switch from a raw base model to an RLHF-tuned chat model, your prompting strategy must change completely because instruction-following and safety guards are now baked in. Ignoring this means you might waste engineering time writing complex application logic to fix behaviors that the underlying model was already trained to handle or avoid.
In practice
You rarely run RLHF yourself unless you are training custom weights, but you interact with its results whenever you select a model checkpoint or adjust system prompts. In API platforms, you might see RLHF effects surface as unexpected refusals on sensitive inputs, or as specific formatting styles that you must override with strict JSON mode or few-shot examples.
The tradeoff
RLHF makes models safer and easier to prompt, but it often degrades raw reasoning capability and increases the rate of sycophancy, where the model agrees with the user even when the user is wrong.