Hallucinations and fabrication

I don’t like the idea that AIs ‘hallucinate’. It brings to mind absent-mindedness – some emeritus-professor confusion or perhaps a kind of stoner wisdom, with the machine opening the doors of perception beyond mortal limitations. Of course behind the linguistic cover story we know what’s really going on: AIs are fabricating information.

Much of this is due to AI systems’ design. These are probabilistic systems, navigating the speculative territory of confidence intervals and inferred intent. They’re trained on piles of data scraped – or, as we’re learning, stolen – from millions of sources. As far as we know, there’s no substantial effort to sift these sources for credibility or authority. So when you ask an LLM to provide academic references for a claim, say, there’s a chance it will rely on false data or simply fill in the gaps to provide a plausible answer.

But I think there’s more to it – and this word ‘plausible’ is key.

As we hopefully realise by now, products take the shape of their creators’ values. Sometimes this happens deliberately but more often it happens unconsciously; a large part of responsible tech work is making this value selection a conscious act.

So while fabrication is an architectural by-product, it’s also a consequence of the values big AI companies have chosen to promote and deprioritise. Companies accept the design’s inherent limitations because they don’t care much about those limitations, at least when compared to the valuable upsides.

The erratic nature of modern gen-AI is a natural result of companies valuing plausibility over accuracy. These systems could be programmed to be honest about their limitations, earning trust through their humility. But instead we often see performative (and false) omniscience.

Why? Because in explosive market phases, potential sells. It gets the world talking, secures investment, and inflates a company’s stock price. Fabrication, then, is a price these companies feel is worth paying to generate this potential. But when trust is collapsing around us, when our epistemic foundations have crumbled and mis/disinformation is fertilising the soil of right-wing populism, fabrication may be the last thing the world needs. We can start fighting it by calling it what it is.