PixelProf
Insane compute wasn’t everything. Hinton helped develop the technique which allowed more data to be processed in more layers of a network without totally losing coherence. It was more of a toy before then because it capped out at how much data could be used, how many layers of a network could be trained, and I believe even that GPUs could be used efficiently for ANNs, but I could be wrong on that one.
Either way, after Hinton’s research in ~2010-2012, problems that seemed extremely difficult to solve (e.g., classifying images and identifying objects in images) became borderline trivial and in under a decade ANNs went from being almost fringe technology that many researches saw as being a toy and useful for a few problems to basically dominating all AI research and CS funding. In almost no time, every university suddenly needed machine learning specialists on payroll, and now at about 10 years later, every year we are pumping out papers and tech that seemed many decades away… Every year… In a very broad range of problems.
The 580 and CUDA made a big impact, but Hinton’s work was absolutely pivotal in being able to utilize that and to even make ANNs seem feasible at all, and it was an overnight thing. Research very rarely explodes this fast.
Edit: I guess also worth clarifying, Hinton was also one of the few researching these techniques in the 80s and has continued being a force in the field, so these big leaps are the culmination of a lot of old, but also very recent work.
Lots of good comments here. I think there’s many reasons, but AI in general is being quite hated on. It’s sad to me - pre-GPT I literally researched how AI can be used to help people be more creative and support human workflows, but our pipelines around the AI are lacking right now. As for the hate, here’s a few perspectives:
- Training data is questionable/debatable ethics,
- Amateur programmers don’t build up the same “code muscle memory”,
- It’s being treated as a sole author (generate all of this code for me) instead of like a ping-pong pair programmer,
- The time saved writing code isn’t being used to review and test the code more carefully than it was before,
- The AI is being used for problem solving, where it’s not ideal, as opposed to code-from-spec where it’s much better,
- Non-Local AI is scraping your (often confidential) data,
- Environmental impact of the use of massive remote LLMs,
- Can be used (according to execs, anyways) to replace entry level developers,
- Devs can have too much faith in the output because they have weak code review skills compared to their code writing skills,
- New programmers can bypass their learning and get an unrealistic perspective of their understanding; this one is most egregious to me as a CS professor, where students and new programmers often think the final answer is what’s important and don’t see the skills they strengthen along the way to the answer.
I like coding with local LLMs and asking occasional questions to larger ones, but the code on larger code bases (with these small, local models) is often pretty non-sensical, but improves with the right approach. Provide it documented functions, examples of a strong and consistent code style, write your test cases in advance so you can verify the outputs, use it as an extension of IDE capabilities (like generating repetitive lines) rather than replacing your problem solving.
I think there is a lot of reasons to hate on it, but I think it’s because the reasons to use it effectively are still being figured out.
Some of my academic colleagues still hate IDEs because tab completion, fast compilers, in-line documentation, and automated code linting (to them) means you don’t really need to know anything or follow any good practices, your editor will do it all for you, so you should just use vim or notepad. It’ll take time to adopt and adapt.
Everything changed when I found the most understanding teachers at the end of my school. I switched schools and had a teacher recognize I was smart and bored and distracted, and she tested me out of the classes and let me spend my time on other random things that were tangentially related and still work with the other students. Game changer compared to where I was where I’d get deductions for doing problems early or reading ahead.
The point on the way to many interests and things, and loving yourself beyond the meds, very important! I found o was regulating myself too much for the first while after diagnosing, and the most relaxation wasn’t what people might typically find relaxing, it was letting the (healthy enough) chaos flow in a safer environment than I was previously prepared to setup.
100%. Great way of putting it. I bounce back forth on occasion, but the trend line is always toward accepting that old part of me, and realizing it’s okay to move on because it’s a very closed chapter that’s been outstaying its welcome. Like any death, you still have those same neural patterns, and they’re slowly getting overwritten, and it’s confusing and disorienting when your muscle memory reaches for something and it’s not there.
It’s extra confusing when what’s reached for is that feeling of not grabbing anything, but you do. When you’ve been falling for decades the ground feels weird for a while when you land.
I definitely feel like a big part of what I’ve grieved is the childhood that I never had, moreso than the future I won’t. It was a big relief, and I felt like I could do well and cut myself slack. I’m just trying to do the same with past me; cut myself that slack, give my past self that love and understanding now that I didn’t get then, accept it was a brutal time, and that it was unfair, but that I’ve grown and learned and stopped rejecting that person was me, and we’re doing all right.
I agree to an extent, but also that the parents need to take time to understand how to “gas them up” appropriately. It’s not everyone’s case, but it became very apparent to me when I was young that my parents would cheer me on over anything, and never take any time to learn about the things they were cheering me on over, and that led to disbelieving pretty much any positive feedback from anyone long-term. The only feedback of substance growing up was the very rare negative feedback, because they would only pull it out when they understood it enough to know it needed improving. That, and emphasizing their efforts as the thing to cheer on, not just the end results.
I’ve learned to work through that, and maybe it goes without saying for most people, but being a genuine and substantive cheerleader is important.