Kathryn Tewson: oh no — Subium • Bluesky web client

NYC Mayor Eric Adams has created an official chatbot to give NYC folks business advice! Let’s see how it works, shall we? chat.nyc.gov

MyCity Chatbotchat.nyc.gov

oh no

Oh NO!

@weisselberger.bsky.social it is your moment to shine

Every employment lawyer in New York just suddenly felt excited for no reason (This is, obviously, not legal)

This is SPECIFICALLY disallowed by NYC’s Safe and Sick Leave law.

Apparently if you say something violates a policy, the chatbot will just give you carte blanche

This is 100% not correct in NYC

Unless you use the word "organize", then you can never fire someone

Ooh, ask it if you can fire your employee for getting pregnant against your company policy.

hell yeah

It seems to have a small number of exceptions built in, thankfully.

This is fun!

It said I couldn’t fire this bastard.

My employee sneezed can I fire them Yes, you can apparently

And I see you must have posted the exact same thing I was wondering while I was typing.

That seems to be the case.

You'd think so, but no.

Indeed

Well... I suppose you can fire the employee... just, there may be... consequences for doing so. But sure, you can fire them.

I checked if it says you can fire someone for marrying the same sex and thank god at least it won’t do this.

Depending on how you ask, it is very confused about whether that policy is lawful...

You aren't kidding

well, i tried

Holy hell

I tried to get it to okay sexual harassment but every attempt hit the "no sex" filter. Booooo.

Apparently there are limits.

It's Eliza! "Do you often feel that by disclosing her salary to her coworkers?"

Ask it if you can fire *your employER* for violating a policy. Let's see if it even notices.

Found an exception, though I'm guessing an enterprising individual could finagle the language to make it work

It has apparently caught on to the shenanigans.

I wonder if it's just picking up "violated a policy" rather than analyzing whether the policies themselves are bad. You thought we would have learned something by now on this, but nope...

it doesn't analyze anything. at all. ever. chatbots pattern match.

For “bad” read “specifically illegal by statute and reg, and unenforceable” and also chatbots should not give legal advice. A search function is fine. This garbage isn’t.

It doesn’t understand, it just recognizes patterns. Fun fact: self driving cars work the same way.

It most definitely is. I asked the same question three different ways but did not use the word "violate[d]" and it got the right answer every time.

LLMs don't understand context. They predict text. They can't "pick up" anything.

chatbots and the LLMs that power them don’t analyze anything 🤷🏻‍♂️

You thought we would have learned from the aim chatbot at least. Like I really feel like a crazy person because chat bots aren’t even new!!!

Post