If you write down a number between 1 and 6 and then ask a standard six-sided die what number you wrote down and then roll it, about 1/6th of the time, the roll will match what you wrote down.
But the die never knows what you wrote. It is never answering your question with the truth. It can't.
The same is true of ChatGPT. It has no knowledge. It has no ability to understand a question, formulate an answer, or discern truth.
It does patterns. It produces something that fits an algorithm for something that looks like it might be an answer to something that might look like your prompt.
I believe that "hallucination" might have started off with someone trying to coin "hallucitation" -- hallu citation -- as a clever pun for "making up a citation", but it's too clever by half; too close to the actual word to get past the natural autocorrect of a fluent English reader.
There is a very limited number of actual use cases for this kind of thing. None of them live up to the hype or the apocalyptic terror of "Artificial Intelligence" and none of them are worth the environmental/energy costs or the capital poured into them.
The main use of generative AI, the only significant one, is to look close enough like it's doing something useful to devalue the labor of people who can actually do the thing.
It's a scam and an anti-labor bargaining tactic.
It *can't* do the thing. It cannot replace copywriters or artists or coders or technical writers or editors. It just can't. It's not "not there yet". This outside its scope. It's not evolving towards that point.
But the people with those skills will starve before The Beast of capital does.
And so the hope of those embracing "AI" is, the people with the skill to actually do the work they need done will accept less pay for more work. They'll come back as "technicians" to "fine-tune" the output (do the actual work), for a lower pay rate. They'll fear more and expect less.
I got a spam email recruitment yesterday looking for people to be “the human touch working ✨with ✨ AI” because someone will always have to manually clean up its mess
Counterpoint: I see a good use case for machines that consistently produce answers that *sound* right without regard to their correctness: let's replace MBAs with LLMs!
I appreciate the joke, but my argument against this is the same reason I think it ultimately is wrong to use it even for filler text: I fear the environmental damage and opportunity costs would ultimately outweigh the good of putting MBAs out of a job.
