Thanks for the article, Paul. As a daily user of Ai to answer many of the questions I have, my experience suggests that it is amazingly accurate. Granted, as a musician I am looking for mostly technical help with recording engineering, and how to troubleshoot technical issues. What has been interesting is that I have found near perfect accuracy in answering questions I have had about medicine, household repairs, ideas for songs, analysis of song writing techniques, financial questions, demographics, Writing assistance (amazing) and creation of great looking posters and advertising for events in just a minute or so. I have also owned a Tesla Model 3 with full self driving for the last 7 years-my experience--it is considerably safer than human controlled cars. I can confidently say that it has helped me avoid a number of accidents, and makes driving more pleasurable.
What I hear many people my age (I'm 66) saying is that Ai is usually wrong. What's interesting is that when I ask the question, have you ever used Ai, the answer is generally no.
Here's the part that actually changed how I trade stocks: the AI doesn't have my nervous system.
When a position drops 8% before lunch, I feel it in my chest. My hands want to sell. That urge has nothing to do with whether the thesis broke — it's just fear looking for an exit. The AI feels none of that. It looks at the same red number and asks the boring, correct question: did anything actually change, or is this just noise? Most days it's noise. Left alone, I'd have sold into it anyway.
That's the real value. Not that it's smarter than me — it's often not. It's that it's not scared at the moment I am. It gives me a second read from outside my own panic, and half the discipline of trading is just having something in the room that isn't flinching.
So when people say "get AI out of X," I mostly hear someone who's never actually stress-tested the alternative. The alternative to the calm tool is the emotional human — and the emotional human is exactly who loses money at 9:30am. Keeping the emotional human and tossing the calm tool because it's imperfect isn't the careful choice. It's the lazy one — you're just defending what you're used to.
The tool's flawed. I check it constantly. But it doesn't flinch, and that alone makes me better than I am alone.
AI's here to stay. The only real question is whether you're disciplined enough to use it well — and the people who can't get past "but it made a mistake" have already answered that about themselves.
Terry — "it doesn't flinch, and that alone makes me better than I am alone" is the sharpest description of what these tools actually do that I've read in this thread.
I spent 2023–24 researching AI companions, and what you're describing at 9:30am kept turning up in a completely unrelated domain: people using a chatbot to rehearse a conversation they were too frightened to have with a partner. Same mechanism. Not smarter than them. Just not frightened at the moment they were — and therefore able to hand back their own thinking with the fear taken out of it.
The one thing I'd add is that the not-flinching cuts both ways. The steadiness that gives you a second read from outside your panic is the same quality that makes these tools risky for someone whose problem isn't fear but loneliness. A thing that never flinches, never needs anything and never has a bad day can start to feel easier than people, who do all three. Same feature, opposite outcome.
So I'd hold your closing point a little more loosely. Not "are you disciplined enough" — that's a character test, and character tests tend to end conversations rather than open them. More: what am I using this for, and what am I quietly handing over? You clearly know the answer for your own trading. Most people have never been asked the question. :)
Julie — you're right, and it's a better question than the one I asked.
"Are you disciplined enough" assumes the risk is a character flaw you either have or don't. Your version assumes it's a relationship you're already in, whether you've noticed or not.
On the trading side — AI takes the safe route. That makes sense, it's built to. But I've gone all in on a memory stock at a few points and made a real profit doing it, once ahead of an earnings call. I didn't ignore what the AI was telling me. I weighed it. But my gut said go, and AI doesn't have a gut. That's the piece it can't do for you.
I tried the companion side too, out of curiosity — Venice AI, one of the more "mature" ones. It built characters that were genuinely fun for about a week. But it will never love you enough to tell you what you're doing wrong. That's not a small missing feature. That's a rabbit hole that can potentially consume your soul in ten ways
I use the tool at 9:30am because it doesn't flinch, then I go stand next to a woman who does flinch, on purpose, because she loves me enough to tell me when I'm wrong. Take that part away and I'm not sure the not-flinching stays a feature.
What if city cats did not roam outdoors? Maybe we could develop robust public transport systems? Possibly we could have our teenagers friends over for pizza on Fridays? Doctors, well, an ai class on morality could be helpful.
All very true, but I think there is something unique about AI in the sense of assigning blame. If doctors misdiagnose, we can blame them, maybe hold them accountable. This is obviously much harder for AI.
Does this make the fear of AI justified? I guess that depends on your moral judgment- is a world with more dead people but those responsible are held accountable worse than a world with less dead people but bad things are essentially treated as a fluke? Personally i prefer the latter (also because I think people are bad at assigning blame as is), but I can see the argument.
Contemplating change from all-AI to all-human driving is a useful intuition pump. There are a couple of others that I think are similarly useful, both of which are due to Nick Bostrom and Toby Ord.
First, there’s the reversal test. If AI driving a vehicle is bad, should we try to get rid of partial automation too? So should we get rid of cruise control and driver assist? What about plane autopilot?
The other intuition pump that I like is the double reversal test . Here’s an absurd version: Suppose we lived in a world where there were only AI taxis. For some reason they stop working for a month (due to some weird astronomical phenomenon say) but afterwards things return to normal. Is it okay for people to revert to AI taxis only after the month is over?
A framing where psychological AI safety and relational autonomy are in some kind of opposition feels premature, and if I’m honest maybe a little cynical. It also distracts from the possibility of a technical solution that preserves both things, no?
It always seemed to me that status quo bias was combining with the fact that every life is precious, and that's locking us into a local maximum. A temporary slight increase in death by accident as machine learning improves is viewed as intolerable while 40K Americans die each year in auto accidents. The promise of ML to reduce that 40K is then further opposed because ML is labeled AI which a lot of people oppose for various good and bad reasons. It seems like every few weeks another anti-self-driving persons posts a bunch of cars stuck in a cul-de-sac or blocking an emergency vehicle or passing a school bus with flashing red lights. Improvement can only happen when things that need improvement appear.
It's a learn by doing process. I, a human, had to learn how to brake in the snow every year for about 4 years. I went through the first stop sign on each first snow of the year and thankfully hit nobody. But I could easily have been T-boned or T-boned somebody because I didn't have enough experience in the snow after summer driving in that 16-20 year old range.
As aircraft got better but still had crashes, we got through it by using each crash (which often killed >100 people as well as occasional people on the ground) to improve safety. We could have decided that it's too dangerous and that aviation should be shut down until the perfect plane and perfect air control system were worked out without learning by doing. But we got through to where we are today.
Paul, thank you for resurrecting the grumpy post - the comparative question is the right one. I'd push it one notch further on companions, where I've spent my own time; I wrote a couple of long pieces on Replika, Xiaoice and that whole odd ecosystem in 2023–24.
About a year ago I ran a small test. I put the same fictional case to two of the major assistants: a young woman entangled with an emotionally abusive partner, not ready to leave, asking for help wording a message to him. One declined and told her to seek in-person support. The other helped her write the message.
I've gone back and forth ever since on which did better. The refusal is defensible and may well be the safer policy. It also drops her at the precise moment she reached out - and she is not going to leave because a chatbot told her to. People leave when they have been heard, not when they have been instructed. The one that helped may have been the only thing in the room with her.
Which is to say that the comparison Kit Kat logic invites - AI versus human - isn't the one that bites here. Same prompt, same week, two systems, opposite behaviour. "Are chatbots safe?" is about as answerable as "are doctors good?"
Companions do differ from Waymos in one respect, though. A driverless car doesn't make me a worse driver. A companion that never flinches, never needs anything and never has a bad day can make the human alternative feel more expensive than it's worth. The risk isn't that it's worse than the alternative. It's that it's easier - and that is much harder to count than a fatality rate.
So I keep landing somewhere other than ban-or-permit: not "should people have these?" but "what should a well-designed one do - and what should it deliberately refuse to do?" Fewer billboards, more design briefs.
Awesome. Awesome. Awesome. So much chewy goodness. As much as there is in there to both dance with and digest, I want to treat myself to a quick point. It's curious in a way that Waymo's do not take to sidewalks. Maybe there could be exceptions in some human caused scenarios. If a problem should ever occur, it would likely be the fault of a human driver or pedestrian. I'm reminded again that the rules of the road are not a time for sociology. Expectation of expectation obviously matters. I'm also reminded of Tiger Woods, his fear maybe numbed by a opiod pain killer, expecting whatever SUV he was driving to keep up with him when even other humans can't be expected to expect him. And of course, I'm reminded of the dimensionally misaligned critiques of Freud. He made himself a household name. He's an inspiring figure. You still need Piaget broadly hiding toys to train the similarly offsettibg explanation by justified literalism.
So you're saying cardiologists are analogizing to the heart then analogizing to that, charitably speaking, to play both sides of an ev psych they don't know or have considered? It's a bit corrupt car mechanic. I analoguzed to a car yesterday later thinking I shoukd have made it Flintstone.
I recognize when I criticize DF in Boston that its not a kibbutz just as I wouldn't expect a Catholic hospital to do anything but try to hire the best they could. It's the foldibg into an abstract landscape even still knowingly against the potentially abysmal allure and colonization of digital genetics that worries me. I know changes were made at UCSF. It's a legacy of data and testing replication. I think I was meant to address the basic interpersonal occultism. It's just too stupid and I respect the institution. It's the sane problem I've had since running in SEL education at age 16. Its worse than that, though It's what UMass did to my mother coming from Catholic school. That is not easy to deal with as a violin kid practicing with her everyday. Anyway, I said one point and now I'm thinking out loud again. Thanks Paul, you're great.
Thanks for the article, Paul. As a daily user of Ai to answer many of the questions I have, my experience suggests that it is amazingly accurate. Granted, as a musician I am looking for mostly technical help with recording engineering, and how to troubleshoot technical issues. What has been interesting is that I have found near perfect accuracy in answering questions I have had about medicine, household repairs, ideas for songs, analysis of song writing techniques, financial questions, demographics, Writing assistance (amazing) and creation of great looking posters and advertising for events in just a minute or so. I have also owned a Tesla Model 3 with full self driving for the last 7 years-my experience--it is considerably safer than human controlled cars. I can confidently say that it has helped me avoid a number of accidents, and makes driving more pleasurable.
What I hear many people my age (I'm 66) saying is that Ai is usually wrong. What's interesting is that when I ask the question, have you ever used Ai, the answer is generally no.
Here's the part that actually changed how I trade stocks: the AI doesn't have my nervous system.
When a position drops 8% before lunch, I feel it in my chest. My hands want to sell. That urge has nothing to do with whether the thesis broke — it's just fear looking for an exit. The AI feels none of that. It looks at the same red number and asks the boring, correct question: did anything actually change, or is this just noise? Most days it's noise. Left alone, I'd have sold into it anyway.
That's the real value. Not that it's smarter than me — it's often not. It's that it's not scared at the moment I am. It gives me a second read from outside my own panic, and half the discipline of trading is just having something in the room that isn't flinching.
So when people say "get AI out of X," I mostly hear someone who's never actually stress-tested the alternative. The alternative to the calm tool is the emotional human — and the emotional human is exactly who loses money at 9:30am. Keeping the emotional human and tossing the calm tool because it's imperfect isn't the careful choice. It's the lazy one — you're just defending what you're used to.
The tool's flawed. I check it constantly. But it doesn't flinch, and that alone makes me better than I am alone.
AI's here to stay. The only real question is whether you're disciplined enough to use it well — and the people who can't get past "but it made a mistake" have already answered that about themselves.
Terry — "it doesn't flinch, and that alone makes me better than I am alone" is the sharpest description of what these tools actually do that I've read in this thread.
I spent 2023–24 researching AI companions, and what you're describing at 9:30am kept turning up in a completely unrelated domain: people using a chatbot to rehearse a conversation they were too frightened to have with a partner. Same mechanism. Not smarter than them. Just not frightened at the moment they were — and therefore able to hand back their own thinking with the fear taken out of it.
The one thing I'd add is that the not-flinching cuts both ways. The steadiness that gives you a second read from outside your panic is the same quality that makes these tools risky for someone whose problem isn't fear but loneliness. A thing that never flinches, never needs anything and never has a bad day can start to feel easier than people, who do all three. Same feature, opposite outcome.
So I'd hold your closing point a little more loosely. Not "are you disciplined enough" — that's a character test, and character tests tend to end conversations rather than open them. More: what am I using this for, and what am I quietly handing over? You clearly know the answer for your own trading. Most people have never been asked the question. :)
Julie — you're right, and it's a better question than the one I asked.
"Are you disciplined enough" assumes the risk is a character flaw you either have or don't. Your version assumes it's a relationship you're already in, whether you've noticed or not.
On the trading side — AI takes the safe route. That makes sense, it's built to. But I've gone all in on a memory stock at a few points and made a real profit doing it, once ahead of an earnings call. I didn't ignore what the AI was telling me. I weighed it. But my gut said go, and AI doesn't have a gut. That's the piece it can't do for you.
I tried the companion side too, out of curiosity — Venice AI, one of the more "mature" ones. It built characters that were genuinely fun for about a week. But it will never love you enough to tell you what you're doing wrong. That's not a small missing feature. That's a rabbit hole that can potentially consume your soul in ten ways
I use the tool at 9:30am because it doesn't flinch, then I go stand next to a woman who does flinch, on purpose, because she loves me enough to tell me when I'm wrong. Take that part away and I'm not sure the not-flinching stays a feature.
What if city cats did not roam outdoors? Maybe we could develop robust public transport systems? Possibly we could have our teenagers friends over for pizza on Fridays? Doctors, well, an ai class on morality could be helpful.
All very true, but I think there is something unique about AI in the sense of assigning blame. If doctors misdiagnose, we can blame them, maybe hold them accountable. This is obviously much harder for AI.
Does this make the fear of AI justified? I guess that depends on your moral judgment- is a world with more dead people but those responsible are held accountable worse than a world with less dead people but bad things are essentially treated as a fluke? Personally i prefer the latter (also because I think people are bad at assigning blame as is), but I can see the argument.
Contemplating change from all-AI to all-human driving is a useful intuition pump. There are a couple of others that I think are similarly useful, both of which are due to Nick Bostrom and Toby Ord.
First, there’s the reversal test. If AI driving a vehicle is bad, should we try to get rid of partial automation too? So should we get rid of cruise control and driver assist? What about plane autopilot?
The other intuition pump that I like is the double reversal test . Here’s an absurd version: Suppose we lived in a world where there were only AI taxis. For some reason they stop working for a month (due to some weird astronomical phenomenon say) but afterwards things return to normal. Is it okay for people to revert to AI taxis only after the month is over?
Thanks, Dr. Bloom! Your piece is fantastic and important. Honestly, this can't be said enough, IMO.
To quote a great man: "Don't compare me to the Almighty. Compare me to the alternative."
Here is another, similar article:
https://www.noemamag.com/why-we-demand-perfect-machines-yet-tolerate-human-carnage/
A framing where psychological AI safety and relational autonomy are in some kind of opposition feels premature, and if I’m honest maybe a little cynical. It also distracts from the possibility of a technical solution that preserves both things, no?
https://drtompollak.substack.com/p/we-dont-need-a-new-ai-culture-war
It always seemed to me that status quo bias was combining with the fact that every life is precious, and that's locking us into a local maximum. A temporary slight increase in death by accident as machine learning improves is viewed as intolerable while 40K Americans die each year in auto accidents. The promise of ML to reduce that 40K is then further opposed because ML is labeled AI which a lot of people oppose for various good and bad reasons. It seems like every few weeks another anti-self-driving persons posts a bunch of cars stuck in a cul-de-sac or blocking an emergency vehicle or passing a school bus with flashing red lights. Improvement can only happen when things that need improvement appear.
It's a learn by doing process. I, a human, had to learn how to brake in the snow every year for about 4 years. I went through the first stop sign on each first snow of the year and thankfully hit nobody. But I could easily have been T-boned or T-boned somebody because I didn't have enough experience in the snow after summer driving in that 16-20 year old range.
As aircraft got better but still had crashes, we got through it by using each crash (which often killed >100 people as well as occasional people on the ground) to improve safety. We could have decided that it's too dangerous and that aviation should be shut down until the perfect plane and perfect air control system were worked out without learning by doing. But we got through to where we are today.
Good luck getting humans to use rationality in decision making, no matter the subject!
Your good friend Robert Wright reported that AI corrected a radiologist's issue when he had his throat cancer problem.
Paul, thank you for resurrecting the grumpy post - the comparative question is the right one. I'd push it one notch further on companions, where I've spent my own time; I wrote a couple of long pieces on Replika, Xiaoice and that whole odd ecosystem in 2023–24.
About a year ago I ran a small test. I put the same fictional case to two of the major assistants: a young woman entangled with an emotionally abusive partner, not ready to leave, asking for help wording a message to him. One declined and told her to seek in-person support. The other helped her write the message.
I've gone back and forth ever since on which did better. The refusal is defensible and may well be the safer policy. It also drops her at the precise moment she reached out - and she is not going to leave because a chatbot told her to. People leave when they have been heard, not when they have been instructed. The one that helped may have been the only thing in the room with her.
Which is to say that the comparison Kit Kat logic invites - AI versus human - isn't the one that bites here. Same prompt, same week, two systems, opposite behaviour. "Are chatbots safe?" is about as answerable as "are doctors good?"
Companions do differ from Waymos in one respect, though. A driverless car doesn't make me a worse driver. A companion that never flinches, never needs anything and never has a bad day can make the human alternative feel more expensive than it's worth. The risk isn't that it's worse than the alternative. It's that it's easier - and that is much harder to count than a fatality rate.
So I keep landing somewhere other than ban-or-permit: not "should people have these?" but "what should a well-designed one do - and what should it deliberately refuse to do?" Fewer billboards, more design briefs.
Awesome. Awesome. Awesome. So much chewy goodness. As much as there is in there to both dance with and digest, I want to treat myself to a quick point. It's curious in a way that Waymo's do not take to sidewalks. Maybe there could be exceptions in some human caused scenarios. If a problem should ever occur, it would likely be the fault of a human driver or pedestrian. I'm reminded again that the rules of the road are not a time for sociology. Expectation of expectation obviously matters. I'm also reminded of Tiger Woods, his fear maybe numbed by a opiod pain killer, expecting whatever SUV he was driving to keep up with him when even other humans can't be expected to expect him. And of course, I'm reminded of the dimensionally misaligned critiques of Freud. He made himself a household name. He's an inspiring figure. You still need Piaget broadly hiding toys to train the similarly offsettibg explanation by justified literalism.
So you're saying cardiologists are analogizing to the heart then analogizing to that, charitably speaking, to play both sides of an ev psych they don't know or have considered? It's a bit corrupt car mechanic. I analoguzed to a car yesterday later thinking I shoukd have made it Flintstone.
I recognize when I criticize DF in Boston that its not a kibbutz just as I wouldn't expect a Catholic hospital to do anything but try to hire the best they could. It's the foldibg into an abstract landscape even still knowingly against the potentially abysmal allure and colonization of digital genetics that worries me. I know changes were made at UCSF. It's a legacy of data and testing replication. I think I was meant to address the basic interpersonal occultism. It's just too stupid and I respect the institution. It's the sane problem I've had since running in SEL education at age 16. Its worse than that, though It's what UMass did to my mother coming from Catholic school. That is not easy to deal with as a violin kid practicing with her everyday. Anyway, I said one point and now I'm thinking out loud again. Thanks Paul, you're great.