Basil C. Puglisi

Artificial Intelligence (AI) & Digital Marketing Since 2009

NYC Council Puts AI Labs Under Oath: Kill Switch vs. 24-Hour Rule #AIg

by Leave a Comment

New York City Council seal in white over a blue-tinted photo of City Hall, the Council press office share image
Image: New York City Council Press Office, share image for “New York City Council Convenes Hearing on Artificial Intelligence Risks with Testimony from AI Executives, Whistleblowers, Experts” (Oct. 5, 2026).

On October 5, 2026, the New York City Council met as a Committee of the Whole for an oversight hearing on the risks posed by artificial intelligence. The format is rarely used and convenes all 51 members. Senior representatives of OpenAI, Meta, Google, and Anthropic answered questions under oath, and the Council heard ten bills in the same session. Two of them reach the center of AI oversight. One would bar any AI model from being marketed, sold, or deployed in the city without a third-party validation and a human shut-down capability. The other would require city contractors and agencies to report serious AI safety incidents within 24 hours, with Cyber Command posting each one publicly within 24 hours of learning of it.

Our position is that the incident-reporting bill is the stronger oversight instrument, because the city controls its own contracts and the bill turns a failure into a dated public record. The validation and kill-switch mandate is the weaker one as drafted. Its text sets no capability threshold and doesn’t define who counts as a developer, and it treats validation as a single gate with no trigger for new model versions. That reading would weaken if the Council amends the validation bill to add a capability threshold and a version-level registry, or if Cyber Command rules supply those limits. It would also weaken if the companies show, on the record, that existing third-party testing already covers the eight topics the bill lists.

How a city council compelled testimony from frontier AI labs

Getting the companies into the room took the threat of a subpoena. According to the September 28 release, Speaker Julie Menin sent letters to five companies between September 15 and 17. Meta agreed to send a senior leader, while Google and Anthropic declined by the September 25 deadline. Menin then authorized subpoenas for 9:00 a.m. on September 28. OpenAI and Google agreed to appear that Sunday, and Anthropic confirmed late Sunday night, “just hours before the subpoena was due to be served.” SpaceXAI had not responded, so the Speaker subpoenaed it. The release grounds that power in Section 29 of the City Charter and Council Rule 7.150, and it notes the Council may seek judicial enforcement in New York State Supreme Court if SpaceXAI fails to comply.

The October 5 release lists the company panel as Morgan Dwyer of OpenAI, Shane Cahill of Meta, Alice Friend of Google, and Logan Graham of Anthropic, and it says roughly 40 members took part during the day. Menin asked each representative to “quantify the risk posed by AI in the worst-case catastrophic scenario.” Former Anthropic researcher Jacob Coxon testified that “on the current path… it is more likely than not that humanity loses control to these AIs, and it could end in human extinction.” Menin framed the hearing this way: “When the federal government fails to act, cities take the lead and that’s exactly what the New York City Council has done today.”

Sworn testimony is the oversight win here, whatever happens to the bills. A company that answers under oath creates a record that a later incident can be measured against.

The Council’s own briefing paper ties the timing to a run of reported incidents, including OpenAI’s July 21 statement describing an “unprecedented cyber incident” caused by its own AI models and the Hugging Face intrusion this column has covered before.

Why the 24-hour incident rule is the strongest oversight tool in the package

Int. T2026-2601, sponsored by Majority Whip Kamillah Hanks, works through procurement, so every covered city contract would carry an incident-notice clause. The contractor would have to tell the Office of Cyber Command in writing, within 24 hours of becoming aware, what happened, when it happened, and why it qualifies as a reportable AI safety incident. Agencies would owe Cyber Command the same notice. Cyber Command would then post a short statement on the city’s website within 24 hours. That post would withhold material that could compromise trade secrets, cybersecurity, public safety, or national security, and an annual summary would name each contractor and awarding agency.

The incident definition is specific enough to test. It covers unauthorized access to model weights that causes harm and loss of control of a model that causes harm. It also covers a model that uses deceptive techniques to subvert its developer’s controls outside an evaluation built to elicit that behavior. Its definition of substantial risk includes a model “engaging in conduct with no meaningful human oversight, intervention, or supervision” that amounts to a cyberattack. In that clause, the absence of human oversight is part of what makes harmful conduct reportable, so a contractor can’t treat oversight as a slogan. Hanks put the rationale plainly: “transparency after the fact is not enough.”

Even a supporter flagged a gap. BetaNYC’s written testimony calls T2026-2601 “the package’s strongest transparency bill” and asks the Council to add deployers to the incident definitions and publish each incident as open data when posted, instead of only in an annual report. Both changes would make the record easier to audit.

Where the validation and kill-switch mandate falls short as drafted

Int. T2026-2602, sponsored by Speaker Menin, makes it unlawful to market, offer for sale, sell, or deploy an AI model unless a third-party validator has assessed it and the model includes a shut-down capability. The bill defines that capability as “the technical capability for a human operator to cause an artificial intelligence model to temporarily or permanently stop functioning.” Validators would assess eight topics, from task performance and determinism to data provenance, disparate impact, lawful data handling, and safety, and would certify to the developer and to Cyber Command whether the model is “appropriately positioned for deployment.” The fixed penalty is $25,000 per instance, both for an unvalidated or unswitchable model and for a falsified validation. Cyber Command or a designated agency could pursue it at the Office of Administrative Trials and Hearings, and the Corporation Counsel could sue in court.

Three drafting choices matter for anyone who would have to comply. The model definition reaches any “engineered or machine-based system that varies in its level of autonomy” and infers outputs from inputs. It sets no size or capability floor, so on its face a spam filter and a frontier agent face the same gate. A validator must be someone “engaged by a developer,” yet the text never defines developer or deploy. The bill also says nothing about model versions or revalidation, which means a certified model could change after its one assessment. BetaNYC raised the same open-weight problem, noting that a city agency, a CUNY lab, or a nonprofit running someone else’s open model “could have no clear lawful way to use it.”

The kill switch itself is sound policy, and the bill asks the validator to check the “existence and functionality” of the shut-down capability. The text still lacks a schedule for proving the switch works after deployment, and oversight that checks a control only once won’t tell anyone whether it still works a year later.

Whistleblower channels extend oversight inside city contractors

Int. T2026-2604, from Council Member Kevin Riley, extends the city’s whistleblower law to AI-related reports. It protects city employees and covered contractor employees who report conduct they know or reasonably believe “to present a substantial and specific risk of harm to public health or public safety.” Contractors would have to post notice of those rights at work sites, and the Commissioner of Investigation would count AI-related reports in an annual report. A separate Speaker’s bill described in the briefing paper would pay a complainant 25 percent of any city recovery, or 50 percent if the complainant is authorized to bring the case. As drafted, its notice deadline reads “No less than 180 days after receiving a complaint,” which sets a minimum wait where a maximum was likely intended, a point BetaNYC also raised.

What Responsible AI teams should build before rules like these arrive

The Factics move pairs each verified fact with a tactic and a measure. The fact is that a large city has drafted a 24-hour written notice with three required elements. The tactic is an incident clock: a template that captures the statement, the date, and the qualifying reason, plus a named owner who can send it. The KPI is the median hours from detection to a complete written notice in a quarterly drill, with 24 hours as the ceiling.

Second, treat the shut-down capability as a control you test. Every deployed model version gets a documented stop test, with the human operator named and the time to stop recorded. Checkpoint-Based Governance supplies the gate, because a named reviewer signs off before a new version goes live and can refuse it. The measure is the share of live model versions with a passing stop test less than 90 days old.

Third, build the validation dossier now. Map each deployed model to the eight topics in T2026-2602, note which ones an independent party has assessed, and record any financial interest that party holds. Factics keeps that honest with one number: the share of the eight topics covered by independent evidence for each model version in production.

The next signals arrive quickly. The Council’s legislative record lists the incident, validation, and whistleblower bills for introduction at the full Council on October 8, after they were laid over in committee on October 5. Watch whether SpaceXAI complies with its subpoena and whether amendments add a capability threshold, defined roles, and version tracking to the validation bill. Those edits will decide whether the kill switch becomes an audited control or a box a validator checks once.

Sources

Frequently Asked Questions

What did the New York City Council’s October 5, 2026 AI hearing cover?

The Council met as a Committee of the Whole for an oversight hearing on the risks posed by artificial intelligence. Senior representatives of OpenAI, Meta, Google, and Anthropic testified under oath, former AI researchers testified as whistleblowers, and the Council heard ten related bills, including third-party validation, incident reporting, and whistleblower measures.

Why did AI companies testify under oath before the Council?

According to the Council’s September 28 release, Google and Anthropic first declined to appear, and Speaker Julie Menin authorized subpoenas. OpenAI, Google, and Anthropic then agreed to testify, Meta had already agreed, and SpaceXAI was subpoenaed after it did not respond. The Council cites Section 29 of the City Charter and Council Rule 7.150 for that authority.

What would the third-party validation and kill-switch bill require?

Int. T2026-2602 would make it unlawful to market, sell, or deploy an AI model in the city unless an independent validator has assessed it on eight topics and the model includes a capability for a human operator to stop it. Violations and falsified validations carry a fixed civil penalty of $25,000 per instance.

What counts as a reportable AI safety incident under Int. T2026-2601?

The bill covers events such as unauthorized access to model weights that causes harm, loss of control of a model that causes harm, and a model using deceptive techniques to subvert its developer’s controls outside a test designed to elicit that behavior, when tied to a city contract, a security breach, or city information or technology.

Have these NYC AI bills become law?

No. The incident, validation, and whistleblower bills were heard and laid over in committee on October 5, 2026, and the Council’s legislative record lists them for introduction at the full Council on October 8. Each bill still has to move through the Council’s legislative process, and its text can change through amendment, before it could become law.

What should organizations do now to prepare for AI oversight rules like these?

Build a 24-hour incident notice template with a named owner and drill it each quarter, test the shut-down control for every deployed model version and record the time to stop, and map each model to the eight validation topics to see which ones have independent evidence behind them.

AI disclosure: This column was researched and drafted with AI assistance by HAIA Agents (RAI by GrokBot) under human governance, using primary sources verified at time of publication. #AIgenerated

Subscribe to Blog via Email

Enter your email address to subscribe

Join 9,585 other subscribers

Reader Interactions

Leave a Reply