About

Katten Advisory Urges Named Human Accountability for Autonomous, Self-Improving AI Systems

Published
Score
21

Why it matters

Katten partners Daniel J. Davis and Alexander C. Kim published an advisory on October 23, 2026, arguing that most AI governance frameworks assume a human can observe, evaluate and intervene in a system's behavior, and that this assumption may not hold through the next phase of AI development. Their core recommendations: human accountability cannot be automated, firms should assign clear responsibility for each AI system, and oversight should scale with the system's level of autonomy. They propose the supervisor-naming model used in financial regulation as a template. They also call for controls that go beyond one-time testing, combining continuous monitoring, external review and explicit guardrails against harmful conduct.

The advisory is a set of principles, not a response to a specific rule. No specific legislation tied to it has been identified. Its emphasis on near-term governance reflects a wave of October 2026 research. A RAND Corporation Expert Insights paper by Gopal P. Sarma, Rachel Steratore, Michael Jacob and Sunny D. Bhatt, published October 12, warns that if AI accelerates AI development, capability jumps could come at shorter intervals than institutions can govern. Sakana AI and UC Berkeley published the MASS framework that same month, in which one model serves as executor, optimizer and evaluator. A self-improvement experiment reported October 15 showed one strict long-horizon evaluation rising from 0 to 5 completed tasks out of 46. Skeptics remain. An August 2026 MIT Technology Review piece reported a study finding AI agents cannot yet conduct open-ended AI research. The experiments are narrow, and it is unclear how far their results generalize.

For attorneys, the practical question is who is accountable when an AI system acts. The advisory argues that answer must name a person, not a process. Review AI governance policies to confirm that each system has an assigned responsible owner, that oversight levels match autonomy, and that testing includes ongoing monitoring rather than only pre-deployment review. Clients deploying autonomous or self-directed AI tools should expect these questions from regulators and counterparties. Track the RAND and MASS work, since follow-on results could shift the timeline for when these controls become necessary.

Sources

mail Subscribe to Artificial Intelligence email updates

Primary sources. No fluff. Straight to your inbox.

Also on LawSnap