AI singer-songwriter ‘Anna Indiana’ debuted her first single ‘Betrayed by this Town’ on X, formerly Twitter—and listeners were not too impressed.
The human imagination also involves the phenomenal experience. You do not just record the data coming at you and regurgitate it, you experience it and then your experience further changes the data itself. We call this “subjectivity” and it’s where creativity comes from.
I am not saying that machine creativity is impossible. What I’m saying is these LLMs are not creative because they don’t even know what they’re doing and they don’t even know “they” are doing it. There’s no “there” there. No more creative than rolling dice.
and experience is ongoing learning, so if an LLM were training on things after the pretraining period then that’d allow it to be creative in your definition?
but in that case, what’s the difference between doing that all at once, and doing it over a period of time?
experience is just tweaking your neurons to make new/different connections
Experience is ongoing learning through the subjective self. When you experience the color red you do not just record it with your photoreceptors, and your experience of the color red is different from mine because we don’t just record wavelengths of light. We don’t just continue to learn from continual exposure to new data, we also continue to learn from generating our own data. In this way our subjective experience is qualitative, not simply quantitative. I don’t just see the specific light wavelengths, I experience the “redness” of red.
When LLM is trained on that kind of data it just starts to hallucinate. This is promising! I think the hallucination phenomenon is actually a precursor to creativity and gives us great insights into the nature of subjective experience. In a sense, my phenomenal experience of the color red is actually much like a hallucination where I am also able to experience the color’s “warmth” and “boldness”. Subjectivity.
it’s only qualitative because we don’t understand it
when an LLM “experiences” new data via training, that’s subjective too: it works its way through the network in a manner that’s different depending on what came before it… if different training data came before it, the network would look differently and the data would change the network as a whole in a different way
The same could be said of a lot of creatives. You speak of greater creativity, that which evokes depth and gravity. There is still more shallow creativity. Learning creativity. That which you do before you learn to do better. Kind of what these are doing.
I’m not saying it’s good or bad, though the people who hold the reigns definitely don’t have the best intentions for their use, but underestimating it is the first step to allowing them to run rampant.
“Never attribute to malice that which you can attribute to stupidity” is the slogan of those who do nothing but look down on others… who underestimate the horrible things the “stupid” can do. Don’t assume stupidity just because you don’t like something. It makes it that much easier for it to bite you on the ass in the future.
I don’t think I’d actually call that shallow thought “creativity”.
Think of a word association game. I don’t think the first word that pops up in my head is creative at all, it’s just a thoughtless reaction.
That’s what LLMs are doing. Without that reflection and depth it’s just a direct input->output