Top physicist says chatbots are just âglorified tape recordersâ::Leading theoretical physicist Michio Kaku predicts quantum computers are far more important for solving mankindâs problems.
I call them glorified spread sheets, but I see the correlation to recorders. LLMs, like most âAIsâ before them, are just new ways to do line of best fit analysis.
Thatâs fine. Glorify those spreadsheets. Itâs a pretty major thing to have cracked.
It is. The tokenization and intent processing are the thing that impress me most. Iâve been joking since the 90âs that the most impressive technological innovation shown on Star Trek TNG was computers that understand the intent of instructions. Now we have that⌠mostly.
To counter the grandiose claims that present-day LLMs are almost AGI, people go too far in the opposite direction. Dismissing it as being only âline of best fit analysisâ fails to recognize the power, significance, and difficulty of extracting meaningful insights and capabilities from data.
Aside from the fact that many modern theories in human cognitive science are actually deeply related to statistical analysis and machine learning (such as embodied cognition, Bayesian predictive coding, and connectionism), referring to it as a âlineâ of best fit is disingenuous because it downplays the important fact that the relationships found in these data are not lines, but rather highly non-linear high-dimensional manifolds. The development of techniques to efficiently discover these relationships in giant datasets is genuinely a HUGE achievement in humanityâs mastery of the sciences, as theyâve allowed us to create programs for things it would be impossible to write out explicitly as a classical program. In particular, our current ability to create classifiers and generators for unstructured data like images would have been unimaginable a couple of decades ago, yet weâve already begun to take it for granted.
So while itâs important to temper expectations that we are a long way from ever seeing anything resembling AGI as itâs typically conceived of, oversimplifying all neural models as being âjustâ line fitting blinds you to the true power and generality that such a framework of manifold learning through optimization represents - as it relates to information theory, energy and entropy in the brain, engineering applications, and the nature of knowledge itself.
Ok, itâs a best fit line on an n-dimentional matrix querying a graphdb ;)
My only point is that this isnât AGI and too many people still fail to recognize that. Now people are becoming disillusioned with it because theyâre realizing it isnât actually creative. Itâs still still just a fancy comparison engine. Thatâs not not world changing, but itâs also not Data from Star Trek
I get that, but what Iâm saying is that calling deep learning âjust fancy comparison engineâ frames the concept in an unnecessarily pessimistic and sneery way. Itâs more illuminating to look at the considerable mileage that âjust pattern matchingâ yields, not only for the practical engineering applications, but for the cognitive scientist and theoretician.
Furthermore, what constitutes being âactually creativeâ? Consider DeepMindâs AlphaGo Zero model:
Mok Jin-seok, who directs the South Korean national Go team, said the Go world has already been imitating the playing styles of previous versions of AlphaGo and creating new ideas from them, and he is hopeful that new ideas will come out from AlphaGo Zero. Mok also added that general trends in the Go world are now being influenced by AlphaGoâs playing style.
Professional Go players and champions concede that the model developed novel styles and strategies that now influence how humans approach the game. If that canât be considered a true spark of creativity, what can?
