Senate Bill 1047, dubbed the "Safe and Secure Innovation for Frontier Artificial Intelligence Models Act," represented California's loud foray into artificial intelligence regulation. The bill aimed to establish an extensive framework for overseeing the development and deployment of large-scale AI models (within the state).
Key provisions
- Mandatory safety testing for AI models with training costs exceeding $100 million
- Implementation of "kill switch" capabilities for rapid model shutdown
- Legal liability for developers if their models cause or enable catastrophic harm
- Robust whistleblower protections for employees in the AI industry
- Empowerment of the California Attorney General to enforce the law
- Establishment of a public cloud computing cluster named "CalCompute"
The bill sought to address growing concerns about the potential risks posed by increasingly powerful AI systems while maintaining California's position at the forefront of technological innovation.
The bill’s timeline
February 7, 2024 - Senator Scott Wiener introduces SB-1047
May 2024 - Bill passes the California Senate with an overwhelming 32-1 vote
June-July 2024 - Amendments made in response to industry feedback
August 22, 2024 - Further amendments in the Assembly
August 28, 2024 - Passes California Assembly (48-16)
August 29, 2024 - Receives final approval from the Senate (30-9)
September 29, 2024 - Governor Newsom vetoes SB-1047
Unpacking SB-1047
At its core, SB-1047 targeted what it termed "covered models" - artificial intelligence systems meeting specific criteria. Before January 1, 2027, these criteria include:
- AI models trained using more than 10²⁶ integer or floating point operations AND costing over $100 million to develop, based on average cloud computing costs at the start of training.
- AI models created by fine-tuning a covered model using at least 3 × 10²⁵ integer or floating point operations AND costing over $10 million, based on average cloud computing costs at the start of fine-tuning.
After January 1, 2027, the Government Operations Agency will determine new thresholds for computing power and costs, unless regulations aren't adopted in time, in which case the previous definitions remain in effect.
The bill mandated several key requirements for developers of covered models:
- Pre-deployment safety testing. Developers would need to assess whether their model could "reasonably cause or materially enable a critical harm" before public release or commercial use.
- Full shutdown capability. The ability to promptly enact a complete shutdown of the model and its derivatives if necessary.
- Safety and security protocols. Detailed written protocols specifying protections and procedures to comply with the developer's duty of care.
- Annual third-party audits. Beginning January 1, 2026, developers would need to retain independent auditors to assess compliance with the bill's requirements.
- AI safety incident reporting. Any AI safety incidents affecting covered models would need to be reported to the Attorney General within 72 hours.