Laura K. Nelson: I' looking for example applications of genAI/LLMs in the humanities or social sciences that would be intelligible to undergrads, and maybe ones that could be a framework for an assignment or small project. Not the "generate an essay then critique it" type. Analytical/scholarly applications?

I' looking for example applications of genAI/LLMs in the humanities or social sciences that would be intelligible to undergrads, and maybe ones that could be a framework for an assignment or small project. Not the "generate an essay then critique it" type. Analytical/scholarly applications?

Underwood's blog post on the passage of time in fiction might be great for this. tedunderwood.com/2023/03/19/u...

Using GPT-4 to measure the passage of time in fictiontedunderwood.com Large language models are valuable research assistants, especially when they refuse to follow instructions.

Excellent example

I think this could be even more intriguing for undergraduates: tedunderwood.com/2024/01/05/c...

Can language models predict the next twist in a story?tedunderwood.com While distant reading has taught us a lot about the history of fiction, it hasn't done much yet to explain why we keep turning pages. "Suspense" is the word we use to explain that impulse. But what is...

Perhaps hard for them to implement, though. It was hard for me.

I could think of basic text classification — take a sentence, label it, see what a BERT does with it. Repeat with maybe a dozen sentences, then let the BERT loose on some more. It’s close to the data, a simple task and no chance of hallucinations.

A specific example could be to let a BERT predict sentences from States of the Union whether they’re from a Dem or Rep and let the students think about why it predicts some right and some wrong and let them improve that

This is a good example because there's a natural ground truth. I'm in Canada so I would just need to figure out how to translate it to this political context

My #PhilosophyOfMind students subject the latest free chatbots to validated cognitive tests to argue (based on the results) for (or against) some Microsoft researchers' claims that LLMs achieved "sparks of artificial general intelligence". Students also provide an objection and counterobjection.

Sparks of Artificial General Intelligence: Early experiments with GPT-4doi.org Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our...

Credit for the best parts of this idea go to @cameronbuckner.bsky.social.

I had students use LLMs to generate potential answers to a research question, supporting scholarly sources, and then search actual databases to see if those sources existed, said what the output claimed, and relative citations to gauge centrality to the topic at hand. The LLM failed instructively.

There's no surprise that using an LLM as a search engine, when it is not a search engine, failed. Stop using LLMs as search engines.

That was the exact lesson the students needed.

Unfortunately due to the irrational opposition to this technology, rather than teaching tech literacy on how to use the tools effectively, they often just get shunned, which means that the kids are learning how to use them through trial and error.

I'm ok with them learning when they fails, but only if they first know enough about how they work to understand why - vectors, embeddings, how they predict words, etc. But like Daniel, I really want them to understand the range of positive things we as scholars can do with them.

I wouldn't say most people need to know those technical elements anymore than they need to know how internet technology in general functions. They just need to understand more of the basics and then what they can and cannot do, and how to make them do what you want them to do within their limits.

Current or potential?

Decided to peek at the original topic. A solid potential case would be creating a natural language interface for existing databases in the humanities. They'd kind of be virtual librarians at that point.

One thing to consider when we're talking about any kind of virtual assistant type service is that these assistants can be monitored by humans. So one librarian could act, effectively, and COMFORTABLY, as many librarians, making that they can effectively help many more people without added strain.

Post