even if you imagine generative AI getting better, I don’t really see how it avoids the problem whereby 1. AI results become the best resource for getting info, 2. the websites and other resources AI draws from lose traffic and die off, 3. now lacking good data to draw from, AI degrades
Suspect the outcome will be:
(1) Mostly non-generative results when a high-trust source has a directly responsive answer.
(2) Generative results for questions of high specificity that require interpolation.
(2) is what generative AI is really good at, and solves what was previously unsolvable.
All of the screencaps people are making are super niche questions. "How do I keep cheese from sliding off my pizza?" "How many rocks should I eat a day?" AI overrates shitposts and comedy sites because these highly specific niche questions get asked and answered so rarely.
Right, so that is not interpolation. That is retrieval augmented generation.
By "specific" I don't mean niche, I mean questions that have no specific answers anywhere - including shitposts - the kind where you search google and nothing useful comes up, even when the answer isn't that hard.
The problem here is that there is no way to tell if this is a product of the LLM or a hand-coded catch to handle something that was very embarrassing for OpenAI a year-ish ago. It's a black box.
Not sure where you are getting that, as ChatGPT is down at the moment.
Anyhoo, here is that question plus a slightly more interesting one that I just submitted to Claude. Both answers are correct (and the latter is heavily interpolated from a general understanding of a programming language):
Dude, I only submitted your question as a joke. Obviously it was going to get that correct.
What about the orders of magnitude harder questions that it answered correctly?
1. It's a black box so you don't have any way of knowing what it's drawing from or synthesizing
2. Usually providing nominally functional code segments does not generalize to other fields
