The fact that this has been replicated is amazing!
Stupid question probably - is computing power what is holding back general AI? I’ve not heard that.
What’s holding back AGI is a complete lack of progress toward anything like intelligence. What we have now isn’t intelligent, it’s multi-variable probability.
It’s not that it’s not intelligent, it’s that predictive language models are obviously just one piece of the puzzle, and we’re going to need all the pieces to get to AGI. It’s looking incredibly doable if we figured out how to make something that’s dumb but sounds smarter than most of us already. We just need to connect it to other models that handle other things better.
There is still heat generated by the act of computation itself, unless you use something like reversible computing but I don’t believe there’s any current way to do that.
And even then, superconducting semiconductors are still going to be some ways off. We could have superconductors for the next decade in power transmission and still have virtually no changes to processesors. I don’t doubt that we will eventually do something close to what you describe, but I’d say it’s easily a long way off still. We’ll probably only be seeing cheaper versions of things that already use superconductors, like MRI machines.
Really appreciate the write up! I didn’t know the computing power required!
Another stupid question (if you don’t mind) - adding superconductors to GPUs doesn’t really se like it would make a huge difference on the heat generation. Sure, some of the heat generated is through trace resistance, but the overwhelming majority is the switching losses of the transistors which will not be effected by superconductor technology. Are we assuming these superconductors will be able to replace semiconductors too? Where are these CPU/GPU efficiencies coming from?