There is a well-known technique that works remarkably well when you want to say one thing and do another: compartmentalization.
Compartmentalization means taking a particular belief, detaching it from the rest of one's beliefs and from one's actual way of life, and placing it in a separate mental compartment. By doing so, the belief becomes effectively immunized: it no longer has to confront other beliefs, experience, or the practical consequences of one's own behavior.
Once a belief has been immunized in this way, it becomes much easier to turn it into dogma. It no longer needs to withstand contradiction or pass the test of reality.
There is one further step. Compartmentalized and dogmatized beliefs can eventually generate an industry around themselves—both a political and an economic one. Many people make a living producing speeches, commentary, and narratives of doom and apocalypse. Once such an industry exists, there are powerful incentives to preserve the beliefs and anxieties on which it depends.
The result is something we see remarkably often: people passionately defending causes that bear little relation to the way they conduct the rest of their lives. Their professed beliefs and their actual behavior inhabit separate compartments.
There is an old and rather simpler word for this: hypocrisy.
Appreciate the compartmentalization framing, but I'd push back gently on "hypocrisy" as the word for it.
Hypocrisy implies the person knows the belief and the behavior don't match and performs anyway.
Simler and Hanson's self-deception argument in Elephant in the Brain makes a stranger claim: the part of the mind doing the talking genuinely doesn't have access to the part optimizing for local incentives. Nobody is running this as a deliberate technique.
That's what makes it worse than hypocrisy, not better. A hypocrite could stop if caught out.
Self-deception doesn't feel like anything from the inside, which is exactly why the doomer can laugh about his children's extinction without a flicker of contradiction.
Personally I am becoming the opposite of the people described here. Rationally I think I believe we'll muddle through. But I find myself behaving as if we won't. Instead of worrying about travel because of my carbon footprint, I fret over how I'd get back if the electric grid collapses; instead of cutting my shower short to save water and energy, I think "aw, enjoy it as long as you want, before hot water stops being a part of life"; instead of hesitating over an Amazon purchase, I think, what the hell, this will all be over soon, and click buy it now. Maybe this is a liminal state one occupies as beliefs change. If so I am not experiencing it as top-down beliefs-change-behavior but rather as bottom-up behavior-changes-belief.
2. I don't think I'd get that much more joy out of spending more money, at least not more than I'd lose by being afraid of #1
3. I don't think we're hopelessly doomed, and saving a large amount of money allows for the possibility of donating to political candidates, researchers, or ethical AI companies.
4. There are social scripts people follow and if I stop following them everyone will think I'm weird, which will make my life less enjoyable.
Also, the concept of AI extinction is terrifying. But so is the concept of death. We can still discuss it cheerfully. We become habituated to the many risks and horrors of our world. We're not frozen in that moment of fear we have when we realize that we'll die one day, otherwise we'd never get anything done.
1. You are entirely right, Dr. Bloom. But I think you undersell incentives. In the attention economy, people have to say more and more absurd / extreme things to get eyeballs. Not a recipe for a useful discussion.
“I think many or most people concerned about death by AGI are not just saying ‘we’re doomed, so give up’, they’re saying ‘we’re doomed unless we do something about it, and the only way we’re going to convince people to do something about it is by expressing it as something that we really believe is likely to happen.’
Same with the anti-Trump concern: while moving away might be feasible, it would be preferable to prevent the US from becoming a dictatorship. Speaking up about it as a real concern may contribute to the prevention efforts.”
I generally agree with you, but this is still a terrible take. Especially a year later, after all the risks the AI industry has taken, and all the security failures they've found.
And the whole argument ignores the motivations to act that come from facing danger squarely, in order to confront it and overcome it.
By analogy, if Israelis express serious concern that Muslim hostility is an existential threat to Israel, would you mock them for being ‘Doomers’? Would you expect them to be just give up, stop investing in their pensions, stop having kids, and partying until the end?
Hopefully not. You'd respect their determination to defend themselves as best they can, without losing all semblance of normal life and hope.
I suggest an imagined conversation between the skeptic (let’s say you’re talking to a skeptical EU policymaker) and the doomer at an AI Safety think-tank.
Skeptic: Do you save for your retirement? If so, why would I take your views seriously?
Doomer: Aha! Well more fool you, I’ve actually taken all my retirement savings out, and split it between a debauched farewell tour of all the pleasures of civilisation, a factory for converting petroleum into glucose, and a bunker filled with weapons, tinned goods and oxygen. Oh yes, and I donate half my earnings to Pause AI!
I hesitate to say this, but this is a fun reflection. I had many different reactions along the way. One was, just because all the doom cults of the past and your rabbi back then were clearly wrong (humanity is still here, to the best of my knowledge) doesn't mean the current doomer crop, notwithstanding the misalignment you note, is wrong too. But I hope I'm wrong about that.
I think that doomers, and even pessimists in general, are just neurotics who want to enumerate possible doomsday scenarios so that 1) someone other than them will find a solution so that they don’t have to or 2) they won’t be caught off guard when one of these scenarios actually happens. It’s a cynical way to live, but it’s also hopeful in a way.
99% P(doom) is very rare, at least from what I've seen. And how are you supposed to act if it's 10%? Or 20%? Or even if it's very high, but you feel powerless to do anything about it? Or if your error bars are very wide? Also, even if you are very certain on the percentage, maybe you're a lot fuzzier on the time frame?
I've found it very difficult to navigate the last few years, and so for the most part I've just kept living my life the same way. It's easy to let inertia carry us along. What's changed is a much higher degree of uncertainty about the near future, and that's just not very actionable.
Maybe it is like learned helplessness. The AI will be so advanced and there are so many ways it can kill us that it's pointless to try and do anything against it.
Surely an element of this is the feeling that most individuals have little power to do anything meaningful. And another element is that even if you think AI is very very risky, it’s unclear how to prepare for its worst case scenarios.
what business is it of yours is the end of the universe? you live in Brooklyn!
Well, the universe is expanding.
Literally LOLed at this.
Alphie, the mother and the smoking Dr. -- what a holy trinity :)
There is a well-known technique that works remarkably well when you want to say one thing and do another: compartmentalization.
Compartmentalization means taking a particular belief, detaching it from the rest of one's beliefs and from one's actual way of life, and placing it in a separate mental compartment. By doing so, the belief becomes effectively immunized: it no longer has to confront other beliefs, experience, or the practical consequences of one's own behavior.
Once a belief has been immunized in this way, it becomes much easier to turn it into dogma. It no longer needs to withstand contradiction or pass the test of reality.
There is one further step. Compartmentalized and dogmatized beliefs can eventually generate an industry around themselves—both a political and an economic one. Many people make a living producing speeches, commentary, and narratives of doom and apocalypse. Once such an industry exists, there are powerful incentives to preserve the beliefs and anxieties on which it depends.
The result is something we see remarkably often: people passionately defending causes that bear little relation to the way they conduct the rest of their lives. Their professed beliefs and their actual behavior inhabit separate compartments.
There is an old and rather simpler word for this: hypocrisy.
Appreciate the compartmentalization framing, but I'd push back gently on "hypocrisy" as the word for it.
Hypocrisy implies the person knows the belief and the behavior don't match and performs anyway.
Simler and Hanson's self-deception argument in Elephant in the Brain makes a stranger claim: the part of the mind doing the talking genuinely doesn't have access to the part optimizing for local incentives. Nobody is running this as a deliberate technique.
That's what makes it worse than hypocrisy, not better. A hypocrite could stop if caught out.
Self-deception doesn't feel like anything from the inside, which is exactly why the doomer can laugh about his children's extinction without a flicker of contradiction.
Personally I am becoming the opposite of the people described here. Rationally I think I believe we'll muddle through. But I find myself behaving as if we won't. Instead of worrying about travel because of my carbon footprint, I fret over how I'd get back if the electric grid collapses; instead of cutting my shower short to save water and energy, I think "aw, enjoy it as long as you want, before hot water stops being a part of life"; instead of hesitating over an Amazon purchase, I think, what the hell, this will all be over soon, and click buy it now. Maybe this is a liminal state one occupies as beliefs change. If so I am not experiencing it as top-down beliefs-change-behavior but rather as bottom-up behavior-changes-belief.
1. I might biff the timing
2. I don't think I'd get that much more joy out of spending more money, at least not more than I'd lose by being afraid of #1
3. I don't think we're hopelessly doomed, and saving a large amount of money allows for the possibility of donating to political candidates, researchers, or ethical AI companies.
4. There are social scripts people follow and if I stop following them everyone will think I'm weird, which will make my life less enjoyable.
Also, the concept of AI extinction is terrifying. But so is the concept of death. We can still discuss it cheerfully. We become habituated to the many risks and horrors of our world. We're not frozen in that moment of fear we have when we realize that we'll die one day, otherwise we'd never get anything done.
1. You are entirely right, Dr. Bloom. But I think you undersell incentives. In the attention economy, people have to say more and more absurd / extreme things to get eyeballs. Not a recipe for a useful discussion.
2. There have been people who really believed it, all through history. (They always insisted "this time is different!" and they've always been wrong. This is a great book about this topic: https://www.goodreads.com/en/book/show/201730880-everything-must-go
This time IS different ;)
Great piece — easy for a non-academic like me to appreciate.
I still believe what I commented last time:
“I think many or most people concerned about death by AGI are not just saying ‘we’re doomed, so give up’, they’re saying ‘we’re doomed unless we do something about it, and the only way we’re going to convince people to do something about it is by expressing it as something that we really believe is likely to happen.’
Same with the anti-Trump concern: while moving away might be feasible, it would be preferable to prevent the US from becoming a dictatorship. Speaking up about it as a real concern may contribute to the prevention efforts.”
I generally agree with you, but this is still a terrible take. Especially a year later, after all the risks the AI industry has taken, and all the security failures they've found.
And the whole argument ignores the motivations to act that come from facing danger squarely, in order to confront it and overcome it.
By analogy, if Israelis express serious concern that Muslim hostility is an existential threat to Israel, would you mock them for being ‘Doomers’? Would you expect them to be just give up, stop investing in their pensions, stop having kids, and partying until the end?
Hopefully not. You'd respect their determination to defend themselves as best they can, without losing all semblance of normal life and hope.
A few months ago, I wrote an article on "doomers acting on their beliefs", suggesting some rational economic options, given doom, and explaining why people tend not to take these options. https://torchestogether.substack.com/p/you-cant-really-bet-on-doom
I suggest an imagined conversation between the skeptic (let’s say you’re talking to a skeptical EU policymaker) and the doomer at an AI Safety think-tank.
Skeptic: Do you save for your retirement? If so, why would I take your views seriously?
Doomer: Aha! Well more fool you, I’ve actually taken all my retirement savings out, and split it between a debauched farewell tour of all the pleasures of civilisation, a factory for converting petroleum into glucose, and a bunker filled with weapons, tinned goods and oxygen. Oh yes, and I donate half my earnings to Pause AI!
They have filled nearly the entire population.
I hesitate to say this, but this is a fun reflection. I had many different reactions along the way. One was, just because all the doom cults of the past and your rabbi back then were clearly wrong (humanity is still here, to the best of my knowledge) doesn't mean the current doomer crop, notwithstanding the misalignment you note, is wrong too. But I hope I'm wrong about that.
How about religious people who think they will go to heaven when they die? Why do they fear death and grieve when loved ones die?
I think that doomers, and even pessimists in general, are just neurotics who want to enumerate possible doomsday scenarios so that 1) someone other than them will find a solution so that they don’t have to or 2) they won’t be caught off guard when one of these scenarios actually happens. It’s a cynical way to live, but it’s also hopeful in a way.
99% P(doom) is very rare, at least from what I've seen. And how are you supposed to act if it's 10%? Or 20%? Or even if it's very high, but you feel powerless to do anything about it? Or if your error bars are very wide? Also, even if you are very certain on the percentage, maybe you're a lot fuzzier on the time frame?
I've found it very difficult to navigate the last few years, and so for the most part I've just kept living my life the same way. It's easy to let inertia carry us along. What's changed is a much higher degree of uncertainty about the near future, and that's just not very actionable.
Maybe it is like learned helplessness. The AI will be so advanced and there are so many ways it can kill us that it's pointless to try and do anything against it.
Surely an element of this is the feeling that most individuals have little power to do anything meaningful. And another element is that even if you think AI is very very risky, it’s unclear how to prepare for its worst case scenarios.
A real doomer just lives their life like everyone else. There isn’t any point doing anything else