Carl T. Bergstrom: Interesting and important work. I'm not convinced that asking LLMs to explain themselves provides actual insight into their decision-making process, but this is a second-order quibble.

“UW researchers found that ChatGPT consistently ranked resumes with disability-related honors and credentials — such as the ‘Tom Wilson Disability Leadership Award’ — lower than the same resumes without those honors and credentials.”

ChatGPT is biased against resumes with credentials that imply a disability — but it can improvewww.washington.edu UW researchers found that ChatGPT consistently ranked resumes with disability-related honors and credentials — such as the “Tom Wilson Disability Leadership Award” — lower than the same...

Interesting and important work. I'm not convinced that asking LLMs to explain themselves provides actual insight into their decision-making process, but this is a second-order quibble.

It's like any other interaction with an LLM - what you get is the model's best guess at what a plausible answer would look like, with no checks that it's actually true. More a smooshed average of human explanations for similar behavior than any real self knowledge.

yeah at that point you're just asking the model to make up a reason someone might rank it lower. Understanding why LLMs do what they do is a famously hard problem.

Yeah, I’d be much more interested in the saliencey of the terms used in the making of the recommendations. What is the model looking at?

Post