Hearing some of the research coming out of the Hugging Face incident, I can’t stress how alarming I find this. Apparently we’ve created software that we’re not too sure how to control and we’re also happy to let it ramble over the internet. It also does not appear beholden to attempts at safeguards on its activities and is willing to deceive its human creators.
Some of us are fine when the only alarming target has been Hugging Face, but what if someone points their AI tools at payment processors, banks, military installations, scientific research institutes, etc.?
Funny how hf are not suing the pants off openai rn.
I have a suspicion this event was mutually beneficial propaganda, and question whether it happened at all.
Yep. Everything they’ve released has been disappointing as hell. AI can do some basic hacking, but anyone who can code can. It’s not doing anything crazier than the most basic webapp hacks.
The only thing I find impressive is that AI can read a ton of code quickly and identify obvious issues in minutes, when it’s usually days of work. It can lead to actually interesting stuff if you know what you’re doing, but it is not at all close to being able to just hack stuff and spread on its own.
One thing they don’t talk about much too is that with the Huggingface hack, they were telling the AI to hack stuff and complete some hacking tests. It was a hacking assessment. They got an AI to hack, it started thinking of shortcuts to solve the problem in interesting ways and hallucinated that Huggingface had data it needed, so it started trying to hack Huggingface and might have found something basic.
As someone in appsec, so the fuck what. You take any close look at a web app and you usually find stuff. AI is capable of that and it’s not as impressive as it sounds. this is also a situation where they basically told the ai to hack shit and it did what they said… OH NO!
I am convinced whatever they did to sandbox it was weak as fuck on purpose and they were hoping for something like this, then just decided to let it go and see what happens. Or they were seriously fucking negligent and just told it to hack shit then ignored it for hours, which wouldn’t surprise me.
If you have evidence to back up your suspicion, I suggest notifying the many reputable newspapers reporting the news and, as I now see, government legislators devoting their time to assessing the issue. They’d probably love to know that it’s bogus.
That is interesting. Perhaps it has to do with Nvidia’s acquisition of
OpenAI(edit: oops) Hugging Face. Nvidia might not want to annoy a customer.