It’s interesting to me how many people I’ve argued with about LLMs. They vehemently insist that this is a world changing technology and the start of the singularity.
Meanwhile whenever I attempt to use one professionally it has to be babied and tightly scoped down or else it goes way off the rails.
And structurally LLMs seem like they’ll always be vulnerable to that. They’re only useful because they bullshit but that also makes them impossible to rely on for anything else.
It’s a computer that understands my words and can reply, even complete tasks upon request, nevermind the result. To me that’s pretty groundbreaking.
It’s a probabilistic network that generates a response based on your input.
No understanding required.
Yet it can outperform humans on some tests involving logic. It will never be perfect, but that implies you can test its IQ
Ask it to write code that replaces every occurrence of “me” in every file name in a folder with “us”, but excluding occurrences that are part of a word (like medium should not be usdium) and it will give you code that does exactly that.
You can ask it to write code that does a heat simulation in a plate of aluminum given one side of heated and the other cooled. It will get there with some help. It works. That’s absolutely fucking crazy.
I’ve been using LLMs pretty extensively in a professional capacity and with the proper grounding work it becomes very useful and reliable.
LLMs on their own is not the world changing tech, LLMs+grounding (what is now being called a Cognitive Architecture), that’s the world changing tech. So while LLMs can be vulnerable to bullshitting, there is a lot of work around them that can qualitatively change their performance.
I’m a few months out of date in the latest in the field and I know it’s changing quickly. What progress has been made towards solving hallucinations? The feeding output into another LLM for evaluation never seemed like a tenable solution to me.
Essentially, you don’t ask them to use their internal knowledge. In fact, you explicitly ask them not to. The technique is generally referred to as Retrieval Augmented Generation. You take the context/user input and you retrieve relevant information from the net/your DB/vector DB/whatever, and you give it to an LLM with how to transform this information (summarize, answer a question, etc).
So you try as much as you can to “ground” the LLM with knowledge that you trust, and to only use this information to perform the task.
So you get a system that can do a really good job at transforming the data you have into the right shape for the task(s) you need to perform, without requiring your LLM to act as a source of information, only a great data massager.
I use chatgpt to make up stuff, imagine things that don’t exist for fun - like a ‘pitch’ for the next new Star Trek series, or to reword my much too succinct prose for a manual for a program I am writing (‘Calcula’ in gitlab) or ideas for a new kind of restaurant (The chef teaches you how to cook the meal you are about to eat) - but never have it code or ask it about facts, it makes them up just as easily as the stuff I just described.
They are useful when you need to generate quasi meaningful bullshit in large volumes easily.
LLMs are being used in medicine now, not to help with diagnosis or correlate seemingly unrelated health data, but to write responses to complaint letters or generate reflective portfolio entries for appraisal.
Don’t get me wrong, outsourcing the bullshit and waffle in medicine is still a win, it frees up time and energy for the highly trained organic general intelligences to do what they do best. I just don’t think it’s the exact outcome the industry expected.
I think it’s the outcome anyone really familiar with the tech expected, but that rarely translates to marketing departments and c-suite types.
I did an LLM project in school, and while that was a limited introduction, it was enough for me to doubt most of the claims coming from LLM orgs. An LLM is good at matching its corpus and that’s about it. So it’ll work well for things like summaries, routine text generation, and similar tasks (and it’s surprisingly good at forming believable text), but it’ll always disappoint with creative work.
I’m sure the tech can do quite a bit more than my class went through, but the limitations here are quite fundamental to the tech.
To me, things like ChatGPT are just more efficient ways to search sites they scrap data from like Stack Overflow. If they ever drive enough traffic away from their sources to kill the sources the likes of ChatGPT will become mostly useless.
I’ve said before it writes like a corporate middle manager.
Not and AI expert but I’ve never been convinced by AI that’s trained on human provided data. It’s just gonna be garbage in, garbage out. To get something substantially useful from AI, it needs to be… axiomatic, I guess. A few years ago, there was Alpha Zero learning only the rules of chess but within just a few hours, it learned all the chess openings/theories that took human chess masters centuries to formulate. It even has it’s own effective opening lines that used to be considered wasteful/unsound before. Granted chess game rules and win conditions are relatively simple compared to real life problems. So may be, it’s too early for general purpose AI research to billions into.
I really liked this article. If you know how AI works, it’s tempting to call what it does “lying”, but I like “bullshit” as a distinct concept for describing being totally indifferent to facts. And it’s interesting that 30% of people in the UK view themselves as having a “bullshit” job, one where they don’t think they contribute anything of value to society. You can totally see why language models would be appealing to so many people.
confabulating
I think this is a better term for it
Edit: ChatGPT == Trump[consummate bullshitter]? I feel like its “intentions” or “programming” are slightly above that but maybe not…
Edit: its interesting how a program can be written that basically replicates Trump’s speech patterns in a replicable and recognizeable way. Where did Trump derive his speech pattern? Like, I know he reads Hitler speeches on his nightstand but is it only Hitler that informs his parole (individual speech patterns)?
The only issue I have there is that it isn’t as widely known and understood. Also if you say someone is confabulating, it means they don’t realize that they are bullshitting, whereas language models are literally designed to bullshit in this manner.
Confabulation
- An informal conversation
- (psychiatry) a plausible but imagined memory that fills in gaps in what is remembered
Like there’s no perfect analogy but I am partial to this characterization, I dunno.
Discuss aha
Edit: I really hope I’m not the originator of this, don’t need that in my life right now. It just seems to fit better imao