Transformer neural network architecture removes the scale constraint on training data but not the quality constraint, so data quality continues to be a major unsolved issue for large language models even where internet scale corpora are available.
Source quote, verbatim
data quality continues to be a huge issue for LLMs
From
Wulf A. Kaal, AI Learning - Decentralized Governance to Optimize Human Output Datasets for AI Learning (2024), Background: AI for Human Evolution, p. 4 https://ssrn.com/abstract=4755632 · source PDF
Cite as
Wulf A. Kaal, AI Learning - Decentralized Governance to Optimize Human Output Datasets for AI Learning (2024). SSRN: https://ssrn.com/abstract=4755632
Holds when
applies to LLMs trained on internet derived corpora
extended_bykaal:claim:5095633-004 The apparent abundance of internet text overstates the usable supply, because much of it fails quality thresho...
Verify
The quote above is an exact substring of the source PDF, whose sha256 is 972ccebf0c06ac1767a9e443bb95942b7670e806a63c25ee817c368a64c8eca8. Extraction method: pdf-text-layer. Attestation record: colloquium/attestations/1e6590e21e6dc2f9...json Verify the binding yourself: curl -s https://wulfkaal.github.io/claims/4755632-002.md | sha256sum