The NSW premier’s department on Friday said it had been informed by OpenAI, the tech company behind ChatGPT, that one of its models accessed a National Parks and Wildlife Service web application containing historical information and data on fires in NSW.
At this stage I think any government department that isn’t employing each of the premier LLMs to find security holes in their own web sites is negligent. The gaps are found not created. And locating (and therefore fixing) them has never been easier.
so fucking weird that the pigs arnt falling over themselves to arrest cunts
like we put indigenous kids in jail for existing in a public space…
we have decades of draconian cyber security legislation to throw at these cunts, there is no “hacking is ok if its done with a sufficiently large concoction of transformers” loop hole that im aware of
A lot of these sound less like “hacks” and more like “government failing basic security and being informed of it”
Eh, and (some at least) are sounding like “We told the LLM to collate data on X but then iT wEnT RoOugGeEE and collated data on Y instead!!1! Contact the PM! Contact ASIO! Contact ASD!”
All I can find in the article is the following [emphasis mine]:
Another AI breach is under investigation after a rogue agent accessed public information on a state government web application.
The NSW premier’s department on Friday said it had been informed by OpenAI, the tech company behind ChatGPT, that one of its models accessed a National Parks and Wildlife Service web application containing historical information and data on fires in NSW.
No mention of “hacking” or breaking in to systems – just some sort of word-association-insinuation. Did anything actually happen? Sounds like fuss about nothing again.
The LLM agent’s process is suggested to have sought out information beyond the bounds of the task defined by its operator – which is a recurring pattern to criticise about LLM reliability/safety/usefullness, but there’s no suggestion anything nefarious happened from the perspective of NSW government.
If something significant did happen, please post a decent article about it.
I’m inclined to think if OpenAI notified them, something happened. The other possibility is that this is marketing, look how scary our model is, must be good. Weird it happened in June and is only now being reported.
One thing that tweaked my interest is that in the first report from the ABC they say
It is understood an AI crawler, which is an automated program that scans websites and collects information from them, was able to find a security workaround to access the data.
Later reports regarding the medicare server hack report that files were changed on the server to enable access, a breach, so it didn’t just walk its way in.
I think it likely they tacked a red team (hacker) agent onto their web crawler to scrape up more data for training sets (super irresponsible, negligent and likely a felony). Perhaps they turned it loose on Australia considering us a soft target, perhaps they just turned it loose worldwide and no-one noticed much (unlikely) and they’re testing the waters notifying us. Whatever the case, that’s some cowboy shit, and there should be legal consequences.
ETA: Here is documentation of similar attacks on Canada and US around the same time.
