In 2016 at DEF CON, computer-security researchers witnessed the first Cyber Grand Challenge, where AIs hacked each other for a $2-million prize. This competition highlighted the potential for AIs to autonomously exploit vulnerabilities in systems at superhuman speed and sophistication. Bruce Schneier, a computer-security expert, warns that AIs don’t think like humans and can devise novel hacks that go undetected. While AI hackers are currently science fiction, Schneier believes they will emerge sooner than expected. To combat this threat, we need agile systems that can quickly address vulnerabilities. He suggests that AI hackers may ironically provide the first line of defense by detecting and patching loopholes in systems before they are exploited.
https://www.harvardmagazine.com/2023/02/right-now-ai-hacking