KICK TECH BROS OUT OF 196
A few points:
None of the big LLMs modify their weights based on input.
So it never learns which is part of intelligence.
Another point:
It has no internal monologue, no private thoughts, no self reflection and no autonomy: it doesn’t exist outside of your function calls, does not (and can not) make guesses about the world and adjust based on those results… Nor do we have a good enough understanding of what’s going on to fix that.
3rd point (example)
Let’s take a hypothetical LLM was given some program to drive it out onto the Internet to learn and fix points 1 and 2. We’ll call the program “consciousness” for the sake of brevity here.
Consciousness comes across a set of references to the latest meme. It queries LLM for what this meme means. LLM will spit out the best statistical match to what it has seen before… But if you’ve ever fed a hot meme into an LLM you’ll know that it’s 90% likely that it will be garbage.
So consciousness now needs to know that the LLM is wrong based on some sort of discrepancy with reality to then teach the LLM, however the only way consciousness can interpret the world is through the LLM, as already established. Consciousness doesn’t know that the LLM doesn’t understand because the LLM will give you a result regardless of if it knows or not. It’s a transformer: it takes inputs and gives outputs. Always.
So we write another layer to make consciousness guess if the LLM is right or not, maybe by having a fuzziness factor output by the LLM to say how hazy its interpretation was. Now consciousness feeds everything about the latest meme into the LLM and asks again and the LLM very confidently responds… With the wrong answer. Because LLM training results are inscrutable (due to the lossy nature of transformation) this will happen eventually, if not every time.
How would consciousness ever define that the LLM had erred?
Human intelligence isn’t just an input output weighted matrix, it’s the interplay of very complex neuronal connections with literal hundreds of types of messages in the brain, all of which modify the nerves every time they’re fired. Sometimes the message from a neuron will be different because the latest input was just enough to bridge that final gap.
An LLM has been trained on vast quantities of data sure, but the data maintained in it’s weights is nowhere near the granularity and quality afforded by actual human cognition. It may have more things stuffed into it that the human mind could ever hold but it lacks the ability of a common rat to interpret anything.
Skipping over the first two points, which I think we’re in agreement on.
To the last, it sounds like you’re saying, “it can’t be intelligent because it is wrong sometimes, and doesn’t have a way to intrinsically know it was wrong.” My argument to that would be, neither do people. When you say something that is incorrect, it requires external input from some other source to alert you to that fact for correction.
That event could then be added to you “training set” as it were, aiding you in not making the mistake in future. The same thing can be done with the AI. That one addition to the training set that was “just enough to bridge that final gap” to the right answer, as it were.
Maybe it’s slower at changing. Maybe it doesn’t make the exact decisions or changes a human would make. But does that mean it’s not “intelligent”? The same might be said for a dolphin or an octopus or an orangutan, all of which are widely considered to be intelligent.
I don’t think they said anything like that “it can’t be intelligent because it’s wrong sometimes”. It’s more like the AI doesn’t exist outside of the prompts you feed it. Humans can introspect, reflect on the actions we’ve done and question what effect our actions had on the situation. Humans can have desires, we can want to be more accurate, truthful in our actions, and reflect on how we might have failed doing this in the past. AI cannot do this. And we can do this outside of the prompt of a similar situation. AI only takes an input and then generates an output, wipes its hands, and calls it a day. It doesn’t matter if it gave you a correct answer, wrong answer, or gave you a completely illegible sentence.
The previous guy and I agreed that you could trivially write a wrapper around it that gives it an internal monologue and feedback loop. So that limitation is artificial and easy to overcome, and has been done in a number of different studies.
And it’s also trivially easy to have the results of its actions go into that feedback loop and influence its weights and models.
And is having wants and desires necessary to be an “intelligence”? That’s getting into the philosophy side of the house, but I would argue that’s superfluous.
My point is not that it can’t be intelligent because it’s wrong sometimes, my point is the program called “consciousness” is what would end up doing all the work we would recognize as sapient.
A LLM is little more than a really weird dictionary that doesn’t let you open it.
The LLM could spit things out and then be retrained but only if “consciousness” can tell it yes or no, feed it info etc.
Currently that consciousness program is entirely computer scientists and there are no promising avenues for replacing them yet.
You missed the point of my “can be wrong” bit. The focus was on the final clause of “and recognize that it was wrong”.
But I’m kinda confused by your last post. You say that only computer scientists are giving it feedback on its “correctness” and therefore it can’t truly be conscious, but that’s trivially untrue and clearly irrelevant.
First, feedback on correctness can be driven by end users. Anyone can tell ChatGPT “I don’t like the way you did that,” and it would be trivially easy to add that to a feedback loop that influences the model over time.
Second, find me a person who’s only feedback loop was internal. People are told “no that’s wrong” or “you’ve messed that up” all the time. That’s what makes us grow as people. That is arguably the core underpinning of what makes something intelligent. The ability to take ideas from other people (computer scientists or no), and have them influence the way you think about things.
Like, it seems like you think that the “consciousness program” you describe would count as an intelligence, but then say it doesn’t because it’s only getting its external information from computer scientists, which seems like a distinction without a difference.
What about this? These weird little dictionaries have lots of emergent properties we’re still exploring.
