Honesty Is the Best Policy—Even for AI
If you reward fluency over truth, don’t be surprised when your AI speaks nonsense beautifully. That is the sobering lesson from recent work on why large language models (LLMs) hallucinate. The research is clear: hallucinations are not mysterious glitches, but the rational outcome of how these systems are trained and evaluated. When the training signal rewards confident answers, models learn to manufacture them—truthful or not. The problem with beautiful nonsense The paper Why Language Models Hallucinate makes a blunt claim: hallucinations arise because LLMs are optimised for being useful and fluent, not necessarily correct. In other words, they are rewarded for looking right more than for being right. That incentive structure guarantees some degree of dishonesty, even if the model has no intention in the human sense. ...