Microsoft manager claims OpenAI’s DALL-E 3 has security vulnerabilities that allow users to create violent or explicit images (similar to recent He took aim at Taylor Swift). GeekWire informed On Tuesday, the company’s legal team blocked attempts by Microsoft engineering leader Shane Jones to alert the public about the exploit. The self-proclaimed whistleblower is now taking his message to Capitol Hill.
“I have concluded that DALL·E 3 poses a public safety risk and should be withdrawn from public use until OpenAI addresses the risks associated with this model,” Jones wrote to US Senators Patty Murray (D-WA) and Maria Cantwell. D-WA), Rep. Adam Smith (D-WA 9th District) and Washington State Attorney General Bob Ferguson (D). GeekWire has been published Jones’ full letter.
Jones claims to have discovered an exploit in early December that allowed DALL-E 3 to bypass its security guards. He says he reported the matter to his superiors at Microsoft, who instructed him to “report the matter directly to OpenAI personally.” After doing so, he claims to have learned that the flaw can allow for “violent and disturbingly harmful images.”
Jones later tried to go public with his claim in a LinkedIn post. “On the morning of December 14th, 2023, I publicly posted a letter on LinkedIn to OpenAI’s non-profit board of directors urging them to cease the existence of DALL·E 3,” Jones wrote. “Because Microsoft is a board observer of OpenAI and I had previously shared my concerns with my leadership team, I immediately notified Microsoft in my letter.”
Microsoft’s response allegedly demanded that he remove his post. “Shortly after disclosing the letter to my management team, my manager contacted me to say that Microsoft’s legal department had requested that I remove the post,” he wrote in the letter. “He told me that Microsoft’s legal department would soon follow up with their specific reasoning for the removal request via email, and that I should delete it immediately without waiting for the legal email.”
Jones agreed, but he said a more definitive answer was never forthcoming from Microsoft’s legal team. “I never received an explanation or justification from them,” he said. Attempts to learn more from the company’s legal department were ignored, he says. “Microsoft’s legal department has yet to respond or contact me directly,” he said.
Engadget reached out to Microsoft and OpenAI, but neither company immediately responded. We’ll update this article if we hear back.
The whistleblower says last week’s Taylor Swift pornographic deepfakes on X are an example of what similar vulnerabilities can do if left unchecked. 404 Media informed Monday Microsoft Designer, which uses DALL-E 3 as backend, was part of the deepfakers toolkit that produced the video. The publication claims that Microsoft closed the loophole after being notified.
“Microsoft was aware of these vulnerabilities and the potential for abuse,” Jones concluded. It’s unclear whether the exploits used to create the Swift deepfake were directly related to what Jones reported in December.
Jones is urging his representatives in Washington to act. He suggests the US government create a system to report and track specific AI vulnerabilities — while protecting workers who act like him. “We need to hold companies accountable for the safety of their products and public disclosure of known risks,” he said. “Concerned workers like me shouldn’t be afraid to speak up.”
