How We Prevent the AI’s from Killing us with Paul Christiano

What Hacker News said

Recommends barking_biscuit · 2023-05 · on “Hugging Face Releases Agents”

>Do you have a counterexample of someone who's gotten their predictions right? Because if not, that should only terrify you even more. If there's anyone out there who predicted how LLMs would work in a way Eliezer failed to, and if that person is predicting "AGI will be cool and will naturally prioritize our well-being", I would love to know. A recent interview with Paul Christiano is about the closest I've come to this. He does note some semi-accurate predictions at the linked timestamp, but the forecast for how things are likely to go is not exactly rosy, though he's quite a bit more optimistic than Eliezer. [this video] Also this whole interview was pretty interesting. Near the end he details

Cites CharlesW · 2023-04 · on ““I think [AI] is the single most likely reason that I will personally die.””

This quote is by Dr. Paul Christiano, PhD Inventor of OpenAI's Deep Reinforcement Learning from Human Preferences (RHLF). Start of thread: [LINK] Full video, "How to Solve AI Alignment with Paul Christiano": [this video]

Cites mitthrowaway2 · 2023-11 · on “Cicero”

I somewhat misremembered; it looks like his point is less about mutual trust and more about supporting whoever has control of reward channels. It starts here where Christiano says that an AI takeover might follow the dynamics of a coup: [this video] And goes into more detail here: [this video] "Suppose that I've been tasked with helping defend you from some other AIs.... My job is, someone is coming to hack your computer and I'm supposed to help defend you. Supposed to help improve your security situation, whatever. And I'm wondering, what is it I could do that will get me a high reward. And one thing I could do that will get me a high reward is actually helping defend your computer, doing the task yo