Post

working in the area of math that led to this tech has been so fucking wild because there's just no way you can trick yourself into believing a lot of linear algebra is going to add up to consciousness unless you have been marinating in the bad brain juices for decades now

I cannot overstate how important it is to understand that these guys are into AI because they’re in a literal eschatological cult

I'm not sure how to explain all this to non-math people but these folks have taken a branch of mathematics and ascribed religiosity to it. "information theory" sounds impressive but I cannot emphasize enough: these are basic functions on distributions. it's all distributions at the bottom of it.

the disconnect is insane. I was at a conference recently where a bunch of academics admitted that the specific extension of info theory intended to work for neuroscience has been a near-total failure and we need to go back to the drawing board. then you read the news and it's like "AGI SOLVED"

bruh there certain kinds of nonlinear informational relationships we can't even detect with a single theory. and then these LLM models aren't even incorporating any of that, they're still doing linear stats, it's just. come on. come the fuck on

then the MOST insane part is that people who know this, who UNDERSTAND THE MATH, still get caught up. that's how seductive this is, apparently

everyone wants it to be true so badly

VCs and Wall Street want it to be true sooo badly because if only they could stop the need to pay human beings to do things, they would all be incrementally richer and the constantly demanding god of shareholder value would be momentarily appeased

i was at a seminar this winter with a director-level FAANG guy in charge of an LLM project. He told us consciousness is just a bunch of linear algebra, literally followed up that he thinks his project is intelligent and more creative than any human (because it can write a shit poem in < 1 s). cringe

have you really found this to be the case? my colleagues just roll their eyes and cash the checks

As someone who is frequently accused of "just knowing math" a lot of these people do not understand it at all and a decent number have never even written a machine learning algorithm, just optimized hyperparameters, parallelism and/or GPU performance

Hi so I also have a phd in physics and am talking about colleagues, thanks.

That was completely fascinating. I want to ask you things but I don’t know what they are yet.

I mean, generating plausible-looking sentences that would fool somebody at least for a bit if they hadn't seen LLM output before is a pretty clever trick for a bunch of equations - it's just not actually intelligence. Making maths write bad poetry is still impressive.

May I ask something? I saw one of these people claim that "AI can create now" even though it's still using the work of humans to do it (insane statement, I know.) As a mathematician, what is your view on this? Can math create something from nothing?

"Create" is a loaded term. Think of it like this: If you draw a bunch of dots on a page, you can draw a line connecting all those dots. The line exists on areas of the page where there was previously no dots. Did it create? There is an equation that represents that line. That equation is the AI.

Yeah I think the thing that weirds me out is that they can do so much with just linear maths but as I said the other day, I would also be impressed by a dog that could say "sausages" but I wouldn't hire it to give an after-dinner talk.

these things are less intelligent than a mosquito. they just stand out because everything else we've accomplished with computers is even fucking stupider!

this was probably what frustrated me the most about the PBS NOVA episode this week - it was clearly meant to be an overview of new tech for the lay person, explaining in basic outlines and showcasing some applications (and some pitfalls/failures). but it kept suggesting it's more than lots of stats!

Here it is on YT if you haven't seen it and would like to check it out - though it's probably region locked to the US. m.youtube.com/watch?v=-sB1...

A.I. Revolution | Full Documentary | NOVA | PBSm.youtube.com Explore the promise and perils of new A.I. technologies. Official Website: https://to.pbs.org/3Py2WDL | #novapbs Can we harness the power of artificial intelligence to solve the world’s most challenging problems without creating an uncontrollable force that ultimately destroys us? ChatGPT and other new A.I. tools can now answer complex questions, write essays, and generate realistic-looking images in a matter of seconds. They can even pass a lawyer’s bar exam. Should we celebrate? Or worry? Or both? Correspondent Miles O’Brien investigates how researchers are trying to transform the world using A.I., hunting for big solutions in fields from medicine to climate change. Chapters 00:00 Introduction 05:38 Modern Origins of Artificial Intelligence 09:05 How Does Artificial Intelligence Work? 16:21 Improving Prosthetics with AI 20:20 Predicting Cancer Diagnosis with AI 25:13 Uses of AI in Medicine 28:48 Other Positive Uses of Artificial Intelligence 37:20 The Danger of AI Fakes 49:23 Conclusion © 2024 WGBH Educational Foundation All rights reserved This program was produced by GBH, which is solely responsible for its content. This program is made possible by viewers like you. Support your local PBS station here: https://pbs.org/donate/ Enjoy full episodes of your favorite PBS shows anytime, anywhere with the free PBS App: https://to.pbs.org/2QbtzhR Stay up to date on the latest science discoveries, full episodes, articles, videos, and more by signing up for NOVA's newsletter here: https://www.pbs.org/wgbh/nova/newsletter/ FOLLOW US: NOVA YouTube: https://www.youtube.com/novaonline Facebook: https://www.facebook.com/NOVApbs X: https://twitter.com/novapbs Instagram: https://www.instagram.com/novapbs/ TikTok: https://www.tiktok.com/@novapbs PBS Facebook: https://www.facebook.com/PBS/ X: https://twitter.com/PBS/ Instagram: https://www.instagram.com/PBS/ YouTube: https://www.youtube.com/PBS/ TikTok: https://www.tiktok.com/@pbs Shop: https://shop.pbs.org/ Artificial intelligence movie, who invented artificial intelligence, what is artificial intelligence, artificial neural network, what is deepfake, how to spot a deepfake, openai, chat gpt #aidocumentary #artificialintelligence #novapbs

As a non math person I find the best way to wrap my head around it is that people are trying to hand society over to a slightly more advanced version of the thing in your phone that was absolutely sure you meant to type "ducking"

Duck yeah.

And critically, when you actually APPLY information theory to these neural networks they turn out to be rather crappy memories and they implement a lossy compression. I have a colleague who did the math and turned it into a textbook: www.amazon.com/Information-...

Information-Driven Machine Learning: Data Science as an Engineering Discipline: Friedland, Gerald: 9783031394768: Amazon.com: Bookswww.amazon.com Information-Driven Machine Learning: Data Science as an Engineering Discipline [Friedland, Gerald] on Amazon.com. *FREE* shipping on qualifying offers. Information-Driven Machine Learning: Data Scienc...

I know some math but zero about this area; would you mind helping me out a bit? Are we talking like probability distributions here, or Schwartz distributions/generalized functions, or...?

It occurs to me that AI will probably kill humanity not because it will become self-aware and destroy us ala SkyNet, but because the LLM is going to give some hapless military person incorrect information and they end up pressing the button that launches the nuclear apocalypse.

WARGAMES (1983)

would you like to play a nice game of chess

lol

I want that RC dinosaur.

"a strange game. the only winning move is to make 5e stat blocks for CR 10 General George Washington."

But hear me out, what if we multiplied a _really big matrix_ of numbers?

"just one more lane" philosophy for a model of consciousness

GOD IT REALLY DOES DRIVE ME INSANE LIKE I LOVE LINEAR ALGEBRA BUT IT'S NOT MAGIC

The entire concept of "I've got it! Linear algebra will save us all!" is such a giant facepalm to me. I know just enough to realize how little of it I know, but there's no way that adds up to "we've recreated brain function". Nope.

The “neurons” in neural networks are basically simplified analogies of Hebbian learning, which is itself a simplified analogy of how only some subpopulations of neurons strengthen connections due to activity, and has little to do with learning on a cognitive or behavioral level.

And this is all a different kind of learning than machine learning, which is just the computer figuring out the parameters of the latent space of whatever it is trained on.

from those of us who work with the brain: THANK YOU FOR SAYING THIS

aka "fun with Markov chains"

But if I do enough maths with a computer it will turn into a girlfriend right?!

Wyatt and Gary proved this in 1985.

bsky.app/profile/oldm...

I'm pretty sure that the application of vast amounts of electricity in the hope of creating a functional brain is a plot point of a well known book by Mary Shelley. These people are incapable of an original idea.

Actually this is an important point. People including myself will express disbelief that mathematicians can believe this sort of thing, but this is the power of centuries of art holding sway over the emotional centers of our brain

Uhhhhhh speak for yourself, my model that finds the optimal amount of milk to add to a bowl of cereal is just one or two constraints from sentience I CAN FEEL IT!!!

Goddamnit, this was a Cryptonomicon reference, wasn’t it? I hate that my brain keeps this sort of shit filed away rather than literally anything useful.

Haha no it was not but now I need to know what that is 🤣

There’s an agonizingly long digression in that book about the ideal amount, temperature and flow speed of milk to add to Captain Crunch, culminating in a design for a spoon that delivers calibrated doses of milk per spoonful. Three guesses at which book Stephenson got too big to edit.

I have the most basic level understanding of how NNs work And it was obvious to me from the start that these things are just amplifying and repeating correlations in the training data

It's just SO natural to say "oh, I'm immune to cognitive bias because I know it exists, so I don't need to check myself" and once you've done that, it's so difficult to recover.

It's not so hard. But this was underway for 20 years or more before anyone in the public ever heard about it

once you convince yourself that mathematics can be a basis for morality all bets are off