# kaal:claim:7456999-022

**Claim.** Arrow's theorem is only an analogy to RLHF aggregation and does not derive reward hacking or overoptimization.

**Type.** methodological  **Support.** argued

**Holds when.**

- the paper's use of Arrow's impossibility theorem

**Source quote.**

> The comparison is an analogy, not an application of the theorem to reward model training and not a derivation of reward overoptimization. Arrow’s theorem does not entail the reward hacking and overoptimization documented by Gao, Schulman, and Hilton (2023).

**From.** Wulf A. Kaal, *Reinforcement Learning from Reputation Feedback* (2026), II.B Analogies: Arrow and Grossman-Hart-Moore, page 8

**Cite as.** Wulf A. Kaal, Reinforcement Learning from Reputation Feedback (2026). SSRN: https://ssrn.com/abstract=7456999

**Verify.** sha256 of source PDF `01016fbbcc25cad205cebd59ea686d8dfee8ae9b632095d86ba6c9a900456a4b` at https://raw.githubusercontent.com/wulfkaal/Academic-Papers/main/papers/pdf/Kaal%20-%202026%20-%20Reinforcement%20Learning%20from%20Reputation%20Feedback.pdf

**Topics.** economics, ai-and-agents, research-methods

**Canonical form.** This markdown file is the canonical hashed representation of the claim. Its sha256 is the content hash used for attestation.