# kaal:claim:7456999-016

**Claim.** Reward-model overoptimization can raise proxy reward while degrading true reward as optimized policies move farther from the reference policy.

**Type.** empirical  **Support.** evidenced

**Holds when.**

- the scaling-law setting documented by Gao, Schulman, and Hilton

**Source quote.**

> Gao, Schulman, and Hilton (2023) demonstrated that reward model overoptimization follows predictable scaling laws: as KL divergence between the optimized policy and the reference policy grows, the proxy reward continues to rise while the true reward degrades.

**From.** Wulf A. Kaal, *Reinforcement Learning from Reputation Feedback* (2026), I. Introduction, page 5

**Cite as.** Wulf A. Kaal, Reinforcement Learning from Reputation Feedback (2026). SSRN: https://ssrn.com/abstract=7456999

**Verify.** sha256 of source PDF `01016fbbcc25cad205cebd59ea686d8dfee8ae9b632095d86ba6c9a900456a4b` at https://raw.githubusercontent.com/wulfkaal/Academic-Papers/main/papers/pdf/Kaal%20-%202026%20-%20Reinforcement%20Learning%20from%20Reputation%20Feedback.pdf

**Topics.** ai-and-agents, risk-and-incentives, empirical-evidence

**Canonical form.** This markdown file is the canonical hashed representation of the claim. Its sha256 is the content hash used for attestation.