# kaal:claim:7456999-024

**Claim.** RLAIF addresses scalability by substituting AI-generated preferences, but it propagates the AI labeler's own biases through the pipeline.

**Type.** failure  **Support.** argued

**Holds when.**

- RLAIF systems whose labeler reflects prior RLHF training

**Source quote.**

> RLAIF (Lee et al. 2024) substitutes AI-generated preferences for human ones, addressing the scalability bottleneck but introducing a new form of the same problem: the AI labeler’s own biases (which are themselves the output of prior RLHF training) propagate through the pipeline.

**From.** Wulf A. Kaal, *Reinforcement Learning from Reputation Feedback* (2026), II.B Analogies: Arrow and Grossman-Hart-Moore, page 8

**Cite as.** Wulf A. Kaal, Reinforcement Learning from Reputation Feedback (2026). SSRN: https://ssrn.com/abstract=7456999

**Verify.** sha256 of source PDF `01016fbbcc25cad205cebd59ea686d8dfee8ae9b632095d86ba6c9a900456a4b` at https://raw.githubusercontent.com/wulfkaal/Academic-Papers/main/papers/pdf/Kaal%20-%202026%20-%20Reinforcement%20Learning%20from%20Reputation%20Feedback.pdf

**Topics.** ai-and-agents, risk-and-incentives, empirical-evidence

**Canonical form.** This markdown file is the canonical hashed representation of the claim. Its sha256 is the content hash used for attestation.