I often see a lot of people with outdated understanding of modern LLMs.
This is probably the best interpretability research to date, by the leading interpretability research team.
It’s worth a read if you want a peek behind the curtain on modern models.
This is a really good science communication article, it describes their work in clear terms (finding structures that relate to abstract concepts, seeing when they are activated and how strengthening and weaking them modifies outputs) and goes into the implications for it. I’m probably going to save this link as a rebuttal for the people who claim LLMs just predict the next word and have no concepts embedded in them.
I doubt that anyone saying that LLM are calculating next word solely based on previous sequence. It’s still statistics, regardless of complexity.
Youd be surprised at the level of unthinking hatred around them, but even discarding that Ive seen it said often that LLMs have no internal model of what they are talking about as they are just next word generators. This quite clearly contradicts that interpretation.
concepts embedded in them
internal model
You used both phrases in this thread, but those are two very different things. It’s a stretch to say this research supports the latter.
Yes, LLMs are still next-token generators. That is a descriptive statement about how they operate. They just have embedded knowledge that allows them to generate sometimes meaningful text.
Yes, but people forget that our brains, and therefore our minds, are also “simply” statistics, albeit very complex.
Saying that it’s “statistics” is, at best, unhelpful. It conveys no useful information. At worst, it’s misleading. What goes on with neural nets has very little to do with what one learns in a stats course.
Yeah, it’s about as useful as saying that all of science is “just statistics”. Which like, in a literal way, it’s true. But science is still what forms the foundation of our entire civilization and base of knowledge.
Knowing that a blood pressure drug works is “just statistics”, but you still take it if your blood pressure is high.
Most people don’t know what Bayesian statistics are so you could say most people don’t really get how machine learning works in general anyway. It’s not misleading though as it perfectly sets expectations on what you’re getting as output. It’s much more healthy to general understanding of AI than anthropomorphizing very inflexible and limited models achieved thanks to technology that is seemingly in a plateau.
It’s a well written article that raises some good points, however it’s also a bit of an ad for their particular ai, the research is practically only within that LLM and every objection about LLM’s it raises, their model seems to be well adjusted or adjustable to it.
I think the most interesting thing in this article is the fact that some concepts central to semantics (analogy, connotation) or psychology (bias) kind of emerge naturally in multi layered neural networks of sufficient size. Also that it can sound like different personalities (overconfident, secretive, delusional) if you manipulate the weight or the proximity of features. I’d like to see the same kind of study but for midjourney…
That’s a chicken and egg situation tho. Is the bias a result of a mind? Or is it the result of being trained on data with common human biases all put together by humans? Are these traits actually measurable or are we just anthropomorphizing a machine like we do everything else?
There is no mind. It’s pretty clear that these people don’t understand their own models. Pretending that there’s a mind and the other absurd anthropomorphisms doesn’t inspire any confidence. Claude is not a person jfc.
