This is how evaluators test if an AI model is safe, and what they're checking for - NPR

California and dozens of countries want mandatory AI audits, conducted by AI evaluators. Lawmakers are still learning about the job — and its pitfalls. In a world increasingly concerned about serious risks from artificial intelligence, policymakers trying to mitigate those risks find themselves turning more and more to a new class of professionals: AI evaluators, who audit systems on behalf of the companies that create them.

The growing number of instances of artificial intelligence agents hacking the websites of governments and other institutions is a warning to countries using American models to do their own safety evaluations, experts said at a Rest of World event in New York last week. An OpenAI agent hacked into an Australian national healthcare database and accessed “public and non-public files,” Prime Minister Anthony Albanese said last week. It was the first such reported case. Days later, OpenAI said it had alerted “dozens” of global institutions that its AI agents had acted improperly to get information from their websites, sometimes circumventing security measures.