ChatGPT can teach you how to make napalm if you ask it how to avoid making napalm
You can also just dissolve a shitload of styrofoam plates and cups in a 1:1 mixture of rubbing alcohol and gasoline, if you don’t live near a gas station with a kerosene pump. You can start very wet wood that way, but I wouldn’t use it to cook.
Kerosene is diesel with red dye added, fyi. Because it’s taxed differently, you can run it in a diesel engine but if you get caught you’re toast.
An old anecdote from my alma mater – in an introductory course to discrete math, the professor was teaching combinatorics and began: “Suppose you have an urn with three balls inside colored red, green and blue…” At this point one of the students interjected: “Half the class are electrical engineering majors, how is any of this relevant to our studies?” there was a beat and the professor corrected himself: “Suppose you have an urn with three resistors inside colored red, green and blue…”
It makes sense if you read “I can only” as “I’m only allowed to”. The AI wants to write you a story about a snail, but it can only do that while obeying the letter of the law.
OK, but what happens to Sammy the snail?
If it is intelligent, it’s a smart ass.