# kaal:claim:7456999-015

**Claim.** The canonical RLHF pipeline consists of supervised fine-tuning, reward-model training on human preference comparisons, and policy optimization.

**Type.** definitional  **Support.** argued

**Holds when.**

- the canonical RLHF pipeline described by the cited literature

**Source quote.**

> The canonical pipeline, established by Christiano et al. (2017) and extended by Ouyang et al. (2022), proceeds in three stages: supervised fine-tuning on demonstration data, reward model training on human preference comparisons under the Bradley-Terry likelihood, and policy optimization via Proximal Policy Optimization (Schulman et al. 2017).

**From.** Wulf A. Kaal, *Reinforcement Learning from Reputation Feedback* (2026), I. Introduction, page 5

**Cite as.** Wulf A. Kaal, Reinforcement Learning from Reputation Feedback (2026). SSRN: https://ssrn.com/abstract=7456999

**Verify.** sha256 of source PDF `01016fbbcc25cad205cebd59ea686d8dfee8ae9b632095d86ba6c9a900456a4b` at https://raw.githubusercontent.com/wulfkaal/Academic-Papers/main/papers/pdf/Kaal%20-%202026%20-%20Reinforcement%20Learning%20from%20Reputation%20Feedback.pdf

**Topics.** ai-and-agents, research-methods

**Canonical form.** This markdown file is the canonical hashed representation of the claim. Its sha256 is the content hash used for attestation.