That's not a huge problem when dealing with a limited corpus. The LLM can piece together words which you can then tokenize and vectorize and then compare to the same info found in the limited dataset it is supposed to talk about. If the LLM goes off into la-la land have it try again.
That’s where I get lost; the part where it lies and you have to say “no you’re lying stop that and try again” which requires that you know when it is lying
If I already know when it’s lying, why did I need to ask in the first place?
Yeah if I was able to just ask jstor to find me evidence supporting my hypothesis that the earth is flat, and then copy garbage it spat out, checked the references it generates to make sure the url loads and call it good I might have graduated having learned literally nothing but to use a chatbot