Why We Urgently Need AI Control Systems

A perpendicular take to AGI fear-mongering

Cassie Kozyrkov
6 min readMay 19, 2024


Approaching the AI control system question from the perspective of fear is clickbait. Talking about “smarter than us” without defining “smarter” is clickbait. Machines have better memory than us and can operate at mind-boggling scale. Already. Today.

And technology is getting more complex rapidly.

So how about we approach the question of control from the perspective of how to make complex tech useful and who should be qualified to steer it?

The control systems we need would help you with AGI (one day) but that is a byproduct. There’s a much better reason to build them.

AI image created by the author.


Recently, tech social media blew up with a discussion of whether we need AI control systems (or whatever you want to call them). Again.

The sensationalist takes could be approximately summarized as:

“We’d better do something urgently so AGI doesn’t outsmart us!”
“AI is still pretty dumb, chill out.”



Cassie Kozyrkov

Chief Decision Scientist, Google. ❤️ Stats, ML/AI, data, puns, art, theatre, decision science. All views are my own.