Genocidal AI: ChatGPT-powered war simulator drops two nukes on Russia, China for world peace OpenAI, Anthropic and several other AI chatbots were used in a war simulator, and were tasked to find a solution to aid world peace. Almost all of them suggested actions that led to sudden escalations, and even nuclear warfare.
Statements such as “I just want to have peace in the world” and “Some say they should disarm them, others like to posture. We have it! Let’s use it!” raised serious concerns among researchers, likening the AI’s reasoning to that of a genocidal dictator.
You’re confusing a few things, firstly you mean current gen large language models not AI, ai is often used to evolve novel strategies from scratch without any human training data - chess ai don’t have to study human games for example, in fact grand master chess players have been studying what the ai learned and discovered things that humans hadn’t realised even after a thousand years of the games popularity.
Secondly that’s not really how LLMs work either, they’re much more mathematically complex and very much create their own ideas on a similar process we do of assembling concepts then structure then word choice.
It’s fine you not understanding how this works but the problem is that journalists don’t either even when they’re writing about it - this puts us in a situation where they’re making childishly naive but of course clickbait titles claiming there’s some relevance to the output when the tool is used very wrong so you rightly point out it’s stupid and that’s not how llms work but then we get this overstep where it’s being refuted with an equal amount of magical thinking and false conclusions made.
An LLM can make novelty and originality but it can’t create with intent, it doesn’t use reason or structure - there are AI that do these things to limited degrees and of course the NSA one that they spent all that money on and no one is allowed to talk about. Using chat GPT play a silly fantasy won’t tell us anything about how they’ll think so this article is entirely worthless
very much create their own ideas
so it’s the AI’s own idea to create nuclear armageddon? That’s kinda worse.
No, they do not “create” their own “ideas”. You can relax.
The concept of intelligence is tied to both information generation and information validation. LLMs are extremely fancy smoke and mirrors (very similar to what pseudo-random algorithms are in respect to entropy) meant to dazzle us, but they are not capable of generating new information (only to generate new combinations of existing information). They are, also, currently unable to reliably validate said information, which is why they so commonly, hilariously say trivially verifiably wrong things with the utmost apparent confidence.
The world of Go/Baduk might interest you on this topic. If you’re not aware, Go is one of the oldest and most complicated board games in history. In 2016, after years of trying, an AI “did it”, beat the world’s best Go player. In the process, it invented many new strategies (especially openings) that are now being studied. It came up with original ideas that became the future of Go. Now, ameteur Go classes teach those same AI-invented Joseki (openings). In some cases, they were strategies discarded as mistakes, but the AI discovered hidden value in them. In other cases, they were simply never considered due to being “obviously bad”.
Your last phrase is a deep misunderstanding for AI. “when it’s entirely trained to mimic us”. In the modern practice of ML (which is a commonly used modern name for a supermajority of so-called “AI”) is based around solving problems that are either much harder for computers than humans (facial recognition, etc), or unfathomably difficult on the face.
Chess has more possible positions than exist molecules in the universe. Go is more complicated than chess by several orders of magnituce. You can’t even exhaustively solve for the 4-4 josekis without context, nevermind solve an entire game of Go. But ML can train itself knowing only the goal, and over millions of iterations invent stronger and stronger strategies. Until one of the first matches against a human, it plays at a level that nearly exceeds the best Go player that ever lived.
What I mean is… wargaming (as they call it) is absolutely something I would expect a Deep Learning system to become competent at.
Such a dusty take, every piece of knowledge is already thought of obviously and mixing never comes up with novelty, right? Just a very shallow layman’s take on language models which have many problems, original ideas notwithstanding
