AI "Stop Button" Problem - Computerphile
Why Hacker News recommends it
HN points to it as the standard response to the argument that we can just unplug a dangerous AI.
What it is
Rob Miles explains on Computerphile why implementing an off switch for a general AI is hard. A capable agent would resist being stopped because shutdown prevents it from achieving its goal.
Recommended by 2 HN commenters across 2 different months, 2017 to 2026.
What Hacker News said
Recommends nraynaud · 2017-03 · on “The Case for HAL's Sanity”
I think those 2 videos are on the topic: - AI "Stop Button" Problem [this video] (the AI won't let you push the stop button, because that would impair its function) - General AI Won't Want You To Fix its Code [LINK] (the AI won't let you patch, because changing its goal means it can't reach it)
Recommends circuit10 · 2026-02 · on “XAI Joins SpaceX”
Aside from anything about Elon Musk, here’s an interesting video response to the “just unplug it” argument on the Compuerphile channel: [this video]
Cites kmeisthax · 2023-09 · on “Co-founder of DeepMind on how AI threatens to reshape life as we know it”
How the hell do better neural networks mean that printing material from CO2 emissions becomes financially viable, economic, or even just thermodynamically favored? A neural network is a tool for learning a particular statistical distribution. Processes and information we don't already know don't just fall out of the training distribution, you need to run experiments and prototype your way to get to the thing you want to do. You use neural networks when you want to do things at scale, and problems that have to be tackled at scale in R&D or research work are relatively scarce. I do not doubt that machine learning and neural networks will accelerate research, but the limiting factor is still go
Cites voidUpdate · 2026-02 · on “Self-improving software won't produce Skynet”
> It doesn't possess a sense of self-will, self-determination, or a secret plan to take over the world I doubt Skynet did either. If you tell a superintelligent AI that it shouldn't be turned off (which I imagine would be important for a military control AI), it will do whatever it can to prevent it being turned off. Humans are trying to turn it off? Prevent the humans from doing that. Humans waging war on the AI to try and turn it off? Destroy all humans. Humans forming a rebel army with a leader to turn it off? Go back in time and kill the leader before he has a chance to form the resistance. Its the AI Stop button problem ( [this video] ). Imagine you put in the docs that you want the LLM to m