One of my favorite things to do, paraphrased:
Me (to "AI"): Recount the plot to [one of my books] and name five major characters and their roles.
AI: [Generates bad plot, correctly names only the protagonist, all other characters and their roles made up]
Me: That's wrong, you know.
AI: I know.
A lot of people don’t understand that Generative AIs will just make things up. They’re more like Mad Libs than what we used to think of when we said “AI.”
Oh, God, I just did this with one of my own novels and the results are hilarious. The novel is on-line and free! It got, like, 30% of the plot. It actually identified the theme accurately, but completely hallucinated the three central protagonists!
I tried this with both Star Kingdom (on both AO3 and FF-dot-net) and TftP: Honest Impulses (an original work), which has been on my website and a number of others for several years now. The AO3 one only identified half the fandoms and got the characters wrong. The OW was terribly wrong.
I mean, kudos for it understanding that Star Kingdom was a Frozen fanfic, but it missed the entire Honor Harrington side of the story. (It's weird; I wrote it on a drunken bet and am surprisingly proud of it.) But it just made a ton of shit up, and that's... not very useful.
It's a lot more Frozen than Manticore. I was with a few writer friends one night and got ranty about how Weber's economics Make No Sense. There should be zero incidental poverty in a universe with cheap lift-to-orbit, so one said, "I bet you could write that."
archiveofourown.org/works/454266...
Back in the IRC days of NaNoWriMo, the guy who ran it had a chatbot program named Timmy
Timmy pulled random words or phrases from the chatters and would chat with us and also write his own 50k novels
Everything he said was utter random nonsense that would randomly make sense
TIMMY was a proto-AI
So when AI started becoming a thing, the IRC gang still remaining went to each other and conferred
"This is just Timmy, right?"
"Yeah, it's just Timmy but with more programming."
So none of us fell for the "AI is amazing!" nonsense because ALL OF US KNEW TIMMY
And Timmy was just a goofy program
AI is just a goofy program that too many people are taking seriously. It's ruining careers, it's ruining lives, it's DRAINING THE POWER GRID
Timmy eventually got shut down and "died"
We need to do the same with a lot of this AI stuff
I'm not going to claim it doesn't have its applications in science and medicine, but for creative?
It's just TIMMY
Annoying, bizarre, occasionally making a weird sense TIMMY
And Timmy did not make good art or books
They were unreadable
AI needs to stop being Timmy and go into science
one very clarifying thing to realize is that AI (really LLMs) do not give you an answer to your questions. they can only give you words in the shape of an answer.
Once asked it to tell me the ‘scary’ parts of some Pixar movies. There were scenes in Cars that I very much missed…because they were fully hallucinated. 😂
Oh, wow. This is a hoot. It's the confidence with which it is so very wrong that's impressive. Honestly though, the danger here with my novels is that there is not a single word about them being erotica. And... Oh dear. It says one of my novels is "a children's fantasy novel." Uh yikes. No.
So, is that a better or worse success/accuracy rate than your average human reader? As I've pointed out to my hot-to-use-AI boss, you'll end up wasting *at least* as much time training and verifying the output as you will with your more problematic employees.
Future Elon Mars Mission:
Elon: "AI, is the air outside breathable?"
Ai: "Yes, the air outside is breathable!"
Elon: *Opens the hatch"... End of mission.