AI & Machine Learning

Understand the machines rewriting everything

Novice

Mastery0%

Novice

Note. AI tools change fast and confidently get things wrong. Verify anything high-stakes against primary sources, and never paste secrets/PII into untrusted endpoints.

Concept 12 of 19

framework

Direct Preference Optimization (DPO)

(Direct Preference Optimization)

Skip the reward model. Teach the LLM straight from "this answer beats that one" pairs.

Definition

A method that fine-tunes a language model directly on pairs of preferred and dispreferred responses, without first training a separate reward model as classic RLHF requires.

Step 1

Predict first

In one or two sentences, what do you think this concept means? Wrong is fine; the small struggle is what makes the answer stick.