Hallucination

When an LLM produces confident, fluent text that's factually wrong or invented — citing fake papers, fabricating quotes, or making up API endpoints.

Hallucination is the LLM tendency to produce text that looks confident and well-formed but is factually wrong. The model doesn't "know" what's true — it predicts what tokens look likely given the prompt, and "likely-looking text about Einstein's 1923 paper" can be invented out of pattern recognition rather than retrieved from any real fact. It matters because hallucinations are the single biggest reason you can't blindly trust LLM output. The text reads fluently, the citations look like real papers, the API endpoints look syntactically valid — but they may be fictional. In legal, medical, financial, or production code contexts, hallucination is a real liability. A notorious example: in 2023 a US lawyer submitted a court brief written by ChatGPT that cited six made-up case names. The cases looked plausible — names, judges, citations all formatted correctly — but didn't exist. The judge sanctioned the lawyer. Mitigations: RAG (ground answers in real documents), tool use (let the model verify with search/databases), prompt techniques ("if you don't know, say so"), and strict output validation (regex / schema checks). Some models are explicitly trained to refuse-when-uncertain, with mixed results. Hallucination rates have dropped substantially in frontier models, but they're not zero. Always verify high-stakes facts. Related: RAG, tool use, alignment, factuality.