California Governor Gavin Newsom has vetoed SB 1047, a bill aimed at preventing bad actors from using artificial intelligence to cause “critical harm” to humans. The California state legislature passed the law on Aug. 28 by a 41-9 vote, but several organizations, including the Chamber of Commerce He urged Newsom to veto the bill. In it veto message On Sept. 29, Newsom said the bill was “well-intentioned” but “fails to consider whether AI systems are deployed in high-risk environments, make critical decisions, or use sensitive data. Instead, the bill imposes strict standards on even the most basic functions.” – as long as a large system accommodates it.”
SB 1047 would make developers of artificial intelligence models responsible for adopting security protocols that would stop catastrophic use of their technology. This includes preventive measures such as testing and assessing external risks, as well as an “emergency stop” that completely shuts down the AI model. The first violation will cost a minimum of $10 million and $30 million for subsequent violations. However, the bill has been reworked to eliminate the state attorney general’s ability to sue negligent AI companies unless a catastrophic event occurs. Companies will only be subject to inventory exemptions and can be sued if their model causes critical harm.
This law will apply to AI models that cost at least $100 million to use and 10^26 FLOPS to train. It would also cover derivative projects where a third party invests $10 million or more to develop or modify the original model. Any company doing business in California will be subject to the regulations if it meets other requirements. Addressing the bill’s focus on large-scale systems, Newsom said, “I don’t believe this is the best approach to protecting the public from the very real threats posed by technology.” Adds a veto message:
By focusing only on the most expensive and large-scale models, SB 1047 creates a regulatory framework to control this fast-moving technology that could give the public a false sense of security. Smaller, specialized models could be the same or more dangerous than those targeted by SB 1047—at the expense of potentially limiting the innovation that drives progress for the benefit of the public good.
An earlier version of SB 1047 would have created a new department called the Border Model Division to oversee and enforce the rules. Instead, the bill was amended before a committee vote to put management in the hands of the Border Models Council, part of the government’s Operations Agency. Nine members will be appointed by the state’s governor and legislature.
The bill faced a complicated road to a final vote. SB 1047 was authored by California State Senator Scott Wiener TechCrunch: “We have a history of waiting for damage to happen and then wringing our hands. Let’s not wait for something bad to happen. Let’s get out ahead of it.” Prominent AI researchers Geoffrey Hinton and Yoshua Bengio sponsored the bill, Center for AI Security raising the alarm On AI risks over the past year.
“Let me be clear — I agree with the author — we cannot wait for a major disaster to occur before we act to protect the public,” Newsom said in his veto message. The statement continues:
California will not abdicate its responsibility. Safety protocols must be adopted. Proactive deterrents must be implemented and severe consequences for bad actors must be clear and enforceable. However, I disagree with a solution that is not informed by an empirical trajectory analysis of AI systems and capabilities to protect public safety. Ultimately, any framework to effectively regulate AI must keep pace with the technology itself.
SB 1047 met with serious resistance from all over the tech space. Researcher Fei-Fei Li criticized bill, as he did Meta Chief AI Scientist Yann LeCun for limiting the potential to explore new uses of AI. A trade group representing tech giants such as Amazon, Apple and Google he said SB 1047 would limit new developments in the state’s technology sector. Venture capital firm Andreeson Horowitz and several startups also questioned whether the bill would place unnecessary financial burdens on artificial intelligence innovators. Anthropic and other opponents of the original bill pushed for amendments has been accepted In the version of SB 1047 that passed the California Appropriations Committee on August 15th.
