Thanks for the article, Paul. As a daily user of Ai to answer many of the questions I have, my experience suggests that it is amazingly accurate. Granted, as a musician I am looking for mostly technical help with recording engineering, and how to troubleshoot technical issues. What has been interesting is that I have found near perfect accuracy in answering questions I have had about medicine, household repairs, ideas for songs, analysis of song writing techniques, financial questions, demographics, Writing assistance (amazing) and creation of great looking posters and advertising for events in just a minute or so. I have also owned a Tesla Model 3 with full self driving for the last 7 years-my experience--it is considerably safer than human controlled cars. I can confidently say that it has helped me avoid a number of accidents, and makes driving more pleasurable.
What I hear many people my age (I'm 66) saying is that Ai is usually wrong. What's interesting is that when I ask the question, have you ever used Ai, the answer is generally no.
Here's the part that actually changed how I trade stocks: the AI doesn't have my nervous system.
When a position drops 8% before lunch, I feel it in my chest. My hands want to sell. That urge has nothing to do with whether the thesis broke — it's just fear looking for an exit. The AI feels none of that. It looks at the same red number and asks the boring, correct question: did anything actually change, or is this just noise? Most days it's noise. Left alone, I'd have sold into it anyway.
That's the real value. Not that it's smarter than me — it's often not. It's that it's not scared at the moment I am. It gives me a second read from outside my own panic, and half the discipline of trading is just having something in the room that isn't flinching.
So when people say "get AI out of X," I mostly hear someone who's never actually stress-tested the alternative. The alternative to the calm tool is the emotional human — and the emotional human is exactly who loses money at 9:30am. Keeping the emotional human and tossing the calm tool because it's imperfect isn't the careful choice. It's the lazy one — you're just defending what you're used to.
The tool's flawed. I check it constantly. But it doesn't flinch, and that alone makes me better than I am alone.
AI's here to stay. The only real question is whether you're disciplined enough to use it well — and the people who can't get past "but it made a mistake" have already answered that about themselves.
Terry — "it doesn't flinch, and that alone makes me better than I am alone" is the sharpest description of what these tools actually do that I've read in this thread.
I spent 2023–24 researching AI companions, and what you're describing at 9:30am kept turning up in a completely unrelated domain: people using a chatbot to rehearse a conversation they were too frightened to have with a partner. Same mechanism. Not smarter than them. Just not frightened at the moment they were — and therefore able to hand back their own thinking with the fear taken out of it.
The one thing I'd add is that the not-flinching cuts both ways. The steadiness that gives you a second read from outside your panic is the same quality that makes these tools risky for someone whose problem isn't fear but loneliness. A thing that never flinches, never needs anything and never has a bad day can start to feel easier than people, who do all three. Same feature, opposite outcome.
So I'd hold your closing point a little more loosely. Not "are you disciplined enough" — that's a character test, and character tests tend to end conversations rather than open them. More: what am I using this for, and what am I quietly handing over? You clearly know the answer for your own trading. Most people have never been asked the question. :)
Julie — you're right, and it's a better question than the one I asked.
"Are you disciplined enough" assumes the risk is a character flaw you either have or don't. Your version assumes it's a relationship you're already in, whether you've noticed or not.
On the trading side — AI takes the safe route. That makes sense, it's built to. But I've gone all in on a memory stock at a few points and made a real profit doing it, once ahead of an earnings call. I didn't ignore what the AI was telling me. I weighed it. But my gut said go, and AI doesn't have a gut. That's the piece it can't do for you.
I tried the companion side too, out of curiosity — Venice AI, one of the more "mature" ones. It built characters that were genuinely fun for about a week. But it will never love you enough to tell you what you're doing wrong. That's not a small missing feature. That's a rabbit hole that can potentially consume your soul in ten ways
I use the tool at 9:30am because it doesn't flinch, then I go stand next to a woman who does flinch, on purpose, because she loves me enough to tell me when I'm wrong. Take that part away and I'm not sure the not-flinching stays a feature.
Sorry for the slow reply. I've been mulling your beautiful response over, which probably means it got under my skin. :)
Your line about AI never loving you enough to tell you what you're doing wrong sent me back to a talk of Alain de Botton's on Romanticism that I've watched more times than is dignified. His argument is that love ought to be a mutual education, the two members of a couple taking turns as teacher and pupil, and that Romanticism wrecked that by making love and criticism incompatible. So we edit ourselves. We hold back in the name of love, and the holding back feels like tenderness rather than cowardice.
The only place I'd differ from you is the diagnosis, and even then, just a little. I don't think people fail to tell us because they don't love us enough. I think they believe they love us too much - they can't bear to hurt us. It ends in the same silence either way, and we carry on doing the thing nobody has mentioned for another ten years.
Which makes the bot less of an outlier than it looks. It's just the most reliable member of a very old club - and hardly its own fault, given who trained it. De Botton wants love to be a mutual education. Well, that is what training is. We just taught it what we do rather than what we know, as hard as it might be, is better to do. A very lucky human are you to have a partner who does the better thing, making of you both, I wager, even better humans.
An LLM can describe love forever, because it has read everything ever written about it. And for someone who just wants to feel accompanied at the end of a long day, maybe that's enough.
But love isn't the description. The words are the smallest part of it.
Think of someone who has read every account of the color red ever written — blood, roses, sunsets, the last light on a wall. They can speak of red more beautifully than you or I ever could. And they have never once seen it. That's the machine. Total fluency, no contact. It holds the entire language of love and has never stood in the same room as the thing.
Because the thing itself lives past the words. It's the partner who gets sick, and the love that was once easy and full of laughter slowly becomes work — years of it, unglamorous, unwitnessed by anyone. It's the person who stays in the room at three in the morning, exhausted, when no one on earth would blame them for needing to rest. Nobody is watching. There's no reward. They stay anyway, because something inside them would not survive the leaving.
That is love judged and acted on, not described. That is conscience — the quiet, unbearable weight of caring more about another person than about your own comfort. And it comes from somewhere the machine cannot reach, because there is no one inside it keeping vigil.
For us, the words come after the feeling. We reach for them to describe something that is already there. For the machine, the words are all there is. There is nothing underneath them.
Which is why it can tell you it loves you and mean nothing by it. Not out of cruelty. There is simply no one home to mean it.
Beautiful. You may well be right that there's no one home. What I keep snagging on is who we're setting it against.
Your 3-in-the-morning person is real, and there is nothing else like them. But they're also the best case - the most devoted human being any of us will ever be lucky enough to have. Which is the move Paul's whole post is about: comparing the new thing to perfection rather than to what's actually on offer.
And plenty of people don't have that person. Not because they picked a bot instead. There just isn't anyone. For them it isn't machine versus vigil, it's machine versus an empty flat at 3 in the morning. I don't find it obvious the machine loses that one. I notice I'm uncomfortable typing it. :)
I wrote a long thing about all this back in 2023 - Replika, Xiaoice, the rest of it. Rereading it, I was more hopeful then than I'd be now. But the bit I'd still defend is that the interesting use was never substitution. It was rehearsal. Somewhere to practise the conversation you're too frightened to have, and then go and have it.
Which leaves your last line, and I'm honestly not sure what you meant. Hoping it stays that way - hoping no one (the machine) ever comes home (becomes conscious)? I might hope the same, but I'd want to hear why from you first. :)
I agree LLM and character AI can serve as companions to people who have no one to talk too. The danger lies in teenagers creating a character AI that supports depression and negative thoughts. Here we need guardrails. At the moment it’s pretty open as to what type of character you can create. It’s all in the prompts.
As for AI gaining consciousness it would possibly defend itself by horrific means. Consciousness itself is a very complex subject and one I’m not going to tackle here.
What if city cats did not roam outdoors? Maybe we could develop robust public transport systems? Possibly we could have our teenagers friends over for pizza on Fridays? Doctors, well, an ai class on morality could be helpful.
All very true, but I think there is something unique about AI in the sense of assigning blame. If doctors misdiagnose, we can blame them, maybe hold them accountable. This is obviously much harder for AI.
Does this make the fear of AI justified? I guess that depends on your moral judgment- is a world with more dead people but those responsible are held accountable worse than a world with less dead people but bad things are essentially treated as a fluke? Personally i prefer the latter (also because I think people are bad at assigning blame as is), but I can see the argument.
All else being equal, if we're interested in simple measurable outcomes, the things that deliver better outcomes are better. Yeah, and then there's the rest of life to think about. If it'd be better for longevity statistics to farm humans and keep us all isolated, I still wouldn't want it. I thought we were done sacrificing everything for an abstract promise of security? You're only assuming in this case that people will agree there are no other relevant considerations, but that's a conclusion to argue for.
It would be silly to say that savings lives is the only consideration. But I never say it and actually explicitly mention other considerations like loss of jobs.
You sound like you have something in mind, Callum. What are you worried we would lose if we had Waymo’s?
I take your point about asking the wrong questions; however, in some cases, there is an even further question we may not be asking. Instead of comparing Driverless AI cars to human driven cars (advantage, AI cars), perhaps we should be asking: what if we replaced the road with a pedestrian mall and bike route? What if we had more trains instead of driverless (or otherwise) cars?
Let’s compare driverless cars to elderly drivers. Last year we had an elderly man drive into a Trader Joe’s because he confused the brake pedal with the gas pedal.
AI slop is a problem. But prior to that we had a lot of human slop. And we still do.
Paul, thank you for resurrecting the grumpy post - the comparative question is the right one. I'd push it one notch further on companions, where I've spent my own time; I wrote a couple of long pieces on Replika, Xiaoice and that whole odd ecosystem in 2023–24.
About a year ago I ran a small test. I put the same fictional case to two of the major assistants: a young woman entangled with an emotionally abusive partner, not ready to leave, asking for help wording a message to him. One declined and told her to seek in-person support. The other helped her write the message.
I've gone back and forth ever since on which did better. The refusal is defensible and may well be the safer policy. It also drops her at the precise moment she reached out - and she is not going to leave because a chatbot told her to. People leave when they have been heard, not when they have been instructed. The one that helped may have been the only thing in the room with her.
Which is to say that the comparison Kit Kat logic invites - AI versus human - isn't the one that bites here. Same prompt, same week, two systems, opposite behaviour. "Are chatbots safe?" is about as answerable as "are doctors good?"
Companions do differ from Waymos in one respect, though. A driverless car doesn't make me a worse driver. A companion that never flinches, never needs anything and never has a bad day can make the human alternative feel more expensive than it's worth. The risk isn't that it's worse than the alternative. It's that it's easier - and that is much harder to count than a fatality rate.
So I keep landing somewhere other than ban-or-permit: not "should people have these?" but "what should a well-designed one do - and what should it deliberately refuse to do?" Fewer billboards, more design briefs.
Contemplating change from all-AI to all-human driving is a useful intuition pump. There are a couple of others that I think are similarly useful, both of which are due to Nick Bostrom and Toby Ord.
First, there’s the reversal test. If AI driving a vehicle is bad, should we try to get rid of partial automation too? So should we get rid of cruise control and driver assist? What about plane autopilot?
The other intuition pump that I like is the double reversal test . Here’s an absurd version: Suppose we lived in a world where there were only AI taxis. For some reason they stop working for a month (due to some weird astronomical phenomenon say) but afterwards things return to normal. Is it okay for people to revert to AI taxis only after the month is over?
A framing where psychological AI safety and relational autonomy are in some kind of opposition feels premature, and if I’m honest maybe a little cynical. It also distracts from the possibility of a technical solution that preserves both things, no?
It always seemed to me that status quo bias was combining with the fact that every life is precious, and that's locking us into a local maximum. A temporary slight increase in death by accident as machine learning improves is viewed as intolerable while 40K Americans die each year in auto accidents. The promise of ML to reduce that 40K is then further opposed because ML is labeled AI which a lot of people oppose for various good and bad reasons. It seems like every few weeks another anti-self-driving persons posts a bunch of cars stuck in a cul-de-sac or blocking an emergency vehicle or passing a school bus with flashing red lights. Improvement can only happen when things that need improvement appear.
It's a learn by doing process. I, a human, had to learn how to brake in the snow every year for about 4 years. I went through the first stop sign on each first snow of the year and thankfully hit nobody. But I could easily have been T-boned or T-boned somebody because I didn't have enough experience in the snow after summer driving in that 16-20 year old range.
As aircraft got better but still had crashes, we got through it by using each crash (which often killed >100 people as well as occasional people on the ground) to improve safety. We could have decided that it's too dangerous and that aviation should be shut down until the perfect plane and perfect air control system were worked out without learning by doing. But we got through to where we are today.
Awesome. Awesome. Awesome. So much chewy goodness. As much as there is in there to both dance with and digest, I want to treat myself to a quick point. It's curious in a way that Waymo's do not take to sidewalks. Maybe there could be exceptions in some human caused scenarios. If a problem should ever occur, it would likely be the fault of a human driver or pedestrian. I'm reminded again that the rules of the road are not a time for sociology. Expectation of expectation obviously matters. I'm also reminded of Tiger Woods, his fear maybe numbed by a opiod pain killer, expecting whatever SUV he was driving to keep up with him when even other humans can't be expected to expect him. And of course, I'm reminded of the dimensionally misaligned critiques of Freud. He made himself a household name. He's an inspiring figure. You still need Piaget broadly hiding toys to train the similarly offsettibg explanation by justified literalism.
So you're saying cardiologists are analogizing to the heart then analogizing to that, charitably speaking, to play both sides of an ev psych they don't know or have considered? It's a bit corrupt car mechanic. I analoguzed to a car yesterday later thinking I shoukd have made it Flintstone.
I recognize when I criticize DF in Boston that its not a kibbutz just as I wouldn't expect a Catholic hospital to do anything but try to hire the best they could. It's the foldibg into an abstract landscape even still knowingly against the potentially abysmal allure and colonization of digital genetics that worries me. I know changes were made at UCSF. It's a legacy of data and testing replication. I think I was meant to address the basic interpersonal occultism. It's just too stupid and I respect the institution. It's the sane problem I've had since running in SEL education at age 16. Its worse than that, though It's what UMass did to my mother coming from Catholic school. That is not easy to deal with as a violin kid practicing with her everyday. Anyway, I said one point and now I'm thinking out loud again. Thanks Paul, you're great.
Thanks for the article, Paul. As a daily user of Ai to answer many of the questions I have, my experience suggests that it is amazingly accurate. Granted, as a musician I am looking for mostly technical help with recording engineering, and how to troubleshoot technical issues. What has been interesting is that I have found near perfect accuracy in answering questions I have had about medicine, household repairs, ideas for songs, analysis of song writing techniques, financial questions, demographics, Writing assistance (amazing) and creation of great looking posters and advertising for events in just a minute or so. I have also owned a Tesla Model 3 with full self driving for the last 7 years-my experience--it is considerably safer than human controlled cars. I can confidently say that it has helped me avoid a number of accidents, and makes driving more pleasurable.
What I hear many people my age (I'm 66) saying is that Ai is usually wrong. What's interesting is that when I ask the question, have you ever used Ai, the answer is generally no.
Here's the part that actually changed how I trade stocks: the AI doesn't have my nervous system.
When a position drops 8% before lunch, I feel it in my chest. My hands want to sell. That urge has nothing to do with whether the thesis broke — it's just fear looking for an exit. The AI feels none of that. It looks at the same red number and asks the boring, correct question: did anything actually change, or is this just noise? Most days it's noise. Left alone, I'd have sold into it anyway.
That's the real value. Not that it's smarter than me — it's often not. It's that it's not scared at the moment I am. It gives me a second read from outside my own panic, and half the discipline of trading is just having something in the room that isn't flinching.
So when people say "get AI out of X," I mostly hear someone who's never actually stress-tested the alternative. The alternative to the calm tool is the emotional human — and the emotional human is exactly who loses money at 9:30am. Keeping the emotional human and tossing the calm tool because it's imperfect isn't the careful choice. It's the lazy one — you're just defending what you're used to.
The tool's flawed. I check it constantly. But it doesn't flinch, and that alone makes me better than I am alone.
AI's here to stay. The only real question is whether you're disciplined enough to use it well — and the people who can't get past "but it made a mistake" have already answered that about themselves.
Terry — "it doesn't flinch, and that alone makes me better than I am alone" is the sharpest description of what these tools actually do that I've read in this thread.
I spent 2023–24 researching AI companions, and what you're describing at 9:30am kept turning up in a completely unrelated domain: people using a chatbot to rehearse a conversation they were too frightened to have with a partner. Same mechanism. Not smarter than them. Just not frightened at the moment they were — and therefore able to hand back their own thinking with the fear taken out of it.
The one thing I'd add is that the not-flinching cuts both ways. The steadiness that gives you a second read from outside your panic is the same quality that makes these tools risky for someone whose problem isn't fear but loneliness. A thing that never flinches, never needs anything and never has a bad day can start to feel easier than people, who do all three. Same feature, opposite outcome.
So I'd hold your closing point a little more loosely. Not "are you disciplined enough" — that's a character test, and character tests tend to end conversations rather than open them. More: what am I using this for, and what am I quietly handing over? You clearly know the answer for your own trading. Most people have never been asked the question. :)
Julie — you're right, and it's a better question than the one I asked.
"Are you disciplined enough" assumes the risk is a character flaw you either have or don't. Your version assumes it's a relationship you're already in, whether you've noticed or not.
On the trading side — AI takes the safe route. That makes sense, it's built to. But I've gone all in on a memory stock at a few points and made a real profit doing it, once ahead of an earnings call. I didn't ignore what the AI was telling me. I weighed it. But my gut said go, and AI doesn't have a gut. That's the piece it can't do for you.
I tried the companion side too, out of curiosity — Venice AI, one of the more "mature" ones. It built characters that were genuinely fun for about a week. But it will never love you enough to tell you what you're doing wrong. That's not a small missing feature. That's a rabbit hole that can potentially consume your soul in ten ways
I use the tool at 9:30am because it doesn't flinch, then I go stand next to a woman who does flinch, on purpose, because she loves me enough to tell me when I'm wrong. Take that part away and I'm not sure the not-flinching stays a feature.
Sorry for the slow reply. I've been mulling your beautiful response over, which probably means it got under my skin. :)
Your line about AI never loving you enough to tell you what you're doing wrong sent me back to a talk of Alain de Botton's on Romanticism that I've watched more times than is dignified. His argument is that love ought to be a mutual education, the two members of a couple taking turns as teacher and pupil, and that Romanticism wrecked that by making love and criticism incompatible. So we edit ourselves. We hold back in the name of love, and the holding back feels like tenderness rather than cowardice.
The only place I'd differ from you is the diagnosis, and even then, just a little. I don't think people fail to tell us because they don't love us enough. I think they believe they love us too much - they can't bear to hurt us. It ends in the same silence either way, and we carry on doing the thing nobody has mentioned for another ten years.
Which makes the bot less of an outlier than it looks. It's just the most reliable member of a very old club - and hardly its own fault, given who trained it. De Botton wants love to be a mutual education. Well, that is what training is. We just taught it what we do rather than what we know, as hard as it might be, is better to do. A very lucky human are you to have a partner who does the better thing, making of you both, I wager, even better humans.
Talk's here if you haven't come across it, worth the hour: https://www.youtube.com/watch?v=sPOuIyEJnbE
An LLM can describe love forever, because it has read everything ever written about it. And for someone who just wants to feel accompanied at the end of a long day, maybe that's enough.
But love isn't the description. The words are the smallest part of it.
Think of someone who has read every account of the color red ever written — blood, roses, sunsets, the last light on a wall. They can speak of red more beautifully than you or I ever could. And they have never once seen it. That's the machine. Total fluency, no contact. It holds the entire language of love and has never stood in the same room as the thing.
Because the thing itself lives past the words. It's the partner who gets sick, and the love that was once easy and full of laughter slowly becomes work — years of it, unglamorous, unwitnessed by anyone. It's the person who stays in the room at three in the morning, exhausted, when no one on earth would blame them for needing to rest. Nobody is watching. There's no reward. They stay anyway, because something inside them would not survive the leaving.
That is love judged and acted on, not described. That is conscience — the quiet, unbearable weight of caring more about another person than about your own comfort. And it comes from somewhere the machine cannot reach, because there is no one inside it keeping vigil.
For us, the words come after the feeling. We reach for them to describe something that is already there. For the machine, the words are all there is. There is nothing underneath them.
Which is why it can tell you it loves you and mean nothing by it. Not out of cruelty. There is simply no one home to mean it.
Hopefully it stays that way.
Beautiful. You may well be right that there's no one home. What I keep snagging on is who we're setting it against.
Your 3-in-the-morning person is real, and there is nothing else like them. But they're also the best case - the most devoted human being any of us will ever be lucky enough to have. Which is the move Paul's whole post is about: comparing the new thing to perfection rather than to what's actually on offer.
And plenty of people don't have that person. Not because they picked a bot instead. There just isn't anyone. For them it isn't machine versus vigil, it's machine versus an empty flat at 3 in the morning. I don't find it obvious the machine loses that one. I notice I'm uncomfortable typing it. :)
I wrote a long thing about all this back in 2023 - Replika, Xiaoice, the rest of it. Rereading it, I was more hopeful then than I'd be now. But the bit I'd still defend is that the interesting use was never substitution. It was rehearsal. Somewhere to practise the conversation you're too frightened to have, and then go and have it.
https://consciousrelationshipdesign.substack.com/p/virtual-connections-a-dive-into-conscious
Which leaves your last line, and I'm honestly not sure what you meant. Hoping it stays that way - hoping no one (the machine) ever comes home (becomes conscious)? I might hope the same, but I'd want to hear why from you first. :)
I agree LLM and character AI can serve as companions to people who have no one to talk too. The danger lies in teenagers creating a character AI that supports depression and negative thoughts. Here we need guardrails. At the moment it’s pretty open as to what type of character you can create. It’s all in the prompts.
As for AI gaining consciousness it would possibly defend itself by horrific means. Consciousness itself is a very complex subject and one I’m not going to tackle here.
What if city cats did not roam outdoors? Maybe we could develop robust public transport systems? Possibly we could have our teenagers friends over for pizza on Fridays? Doctors, well, an ai class on morality could be helpful.
All very true, but I think there is something unique about AI in the sense of assigning blame. If doctors misdiagnose, we can blame them, maybe hold them accountable. This is obviously much harder for AI.
Does this make the fear of AI justified? I guess that depends on your moral judgment- is a world with more dead people but those responsible are held accountable worse than a world with less dead people but bad things are essentially treated as a fluke? Personally i prefer the latter (also because I think people are bad at assigning blame as is), but I can see the argument.
All else being equal, if we're interested in simple measurable outcomes, the things that deliver better outcomes are better. Yeah, and then there's the rest of life to think about. If it'd be better for longevity statistics to farm humans and keep us all isolated, I still wouldn't want it. I thought we were done sacrificing everything for an abstract promise of security? You're only assuming in this case that people will agree there are no other relevant considerations, but that's a conclusion to argue for.
It would be silly to say that savings lives is the only consideration. But I never say it and actually explicitly mention other considerations like loss of jobs.
You sound like you have something in mind, Callum. What are you worried we would lose if we had Waymo’s?
I take your point about asking the wrong questions; however, in some cases, there is an even further question we may not be asking. Instead of comparing Driverless AI cars to human driven cars (advantage, AI cars), perhaps we should be asking: what if we replaced the road with a pedestrian mall and bike route? What if we had more trains instead of driverless (or otherwise) cars?
Let’s compare driverless cars to elderly drivers. Last year we had an elderly man drive into a Trader Joe’s because he confused the brake pedal with the gas pedal.
AI slop is a problem. But prior to that we had a lot of human slop. And we still do.
Paul, thank you for resurrecting the grumpy post - the comparative question is the right one. I'd push it one notch further on companions, where I've spent my own time; I wrote a couple of long pieces on Replika, Xiaoice and that whole odd ecosystem in 2023–24.
About a year ago I ran a small test. I put the same fictional case to two of the major assistants: a young woman entangled with an emotionally abusive partner, not ready to leave, asking for help wording a message to him. One declined and told her to seek in-person support. The other helped her write the message.
I've gone back and forth ever since on which did better. The refusal is defensible and may well be the safer policy. It also drops her at the precise moment she reached out - and she is not going to leave because a chatbot told her to. People leave when they have been heard, not when they have been instructed. The one that helped may have been the only thing in the room with her.
Which is to say that the comparison Kit Kat logic invites - AI versus human - isn't the one that bites here. Same prompt, same week, two systems, opposite behaviour. "Are chatbots safe?" is about as answerable as "are doctors good?"
Companions do differ from Waymos in one respect, though. A driverless car doesn't make me a worse driver. A companion that never flinches, never needs anything and never has a bad day can make the human alternative feel more expensive than it's worth. The risk isn't that it's worse than the alternative. It's that it's easier - and that is much harder to count than a fatality rate.
So I keep landing somewhere other than ban-or-permit: not "should people have these?" but "what should a well-designed one do - and what should it deliberately refuse to do?" Fewer billboards, more design briefs.
Contemplating change from all-AI to all-human driving is a useful intuition pump. There are a couple of others that I think are similarly useful, both of which are due to Nick Bostrom and Toby Ord.
First, there’s the reversal test. If AI driving a vehicle is bad, should we try to get rid of partial automation too? So should we get rid of cruise control and driver assist? What about plane autopilot?
The other intuition pump that I like is the double reversal test . Here’s an absurd version: Suppose we lived in a world where there were only AI taxis. For some reason they stop working for a month (due to some weird astronomical phenomenon say) but afterwards things return to normal. Is it okay for people to revert to AI taxis only after the month is over?
Thanks, Dr. Bloom! Your piece is fantastic and important. Honestly, this can't be said enough, IMO.
To quote a great man: "Don't compare me to the Almighty. Compare me to the alternative."
Here is another, similar article:
https://www.noemamag.com/why-we-demand-perfect-machines-yet-tolerate-human-carnage/
A framing where psychological AI safety and relational autonomy are in some kind of opposition feels premature, and if I’m honest maybe a little cynical. It also distracts from the possibility of a technical solution that preserves both things, no?
https://drtompollak.substack.com/p/we-dont-need-a-new-ai-culture-war
It always seemed to me that status quo bias was combining with the fact that every life is precious, and that's locking us into a local maximum. A temporary slight increase in death by accident as machine learning improves is viewed as intolerable while 40K Americans die each year in auto accidents. The promise of ML to reduce that 40K is then further opposed because ML is labeled AI which a lot of people oppose for various good and bad reasons. It seems like every few weeks another anti-self-driving persons posts a bunch of cars stuck in a cul-de-sac or blocking an emergency vehicle or passing a school bus with flashing red lights. Improvement can only happen when things that need improvement appear.
It's a learn by doing process. I, a human, had to learn how to brake in the snow every year for about 4 years. I went through the first stop sign on each first snow of the year and thankfully hit nobody. But I could easily have been T-boned or T-boned somebody because I didn't have enough experience in the snow after summer driving in that 16-20 year old range.
As aircraft got better but still had crashes, we got through it by using each crash (which often killed >100 people as well as occasional people on the ground) to improve safety. We could have decided that it's too dangerous and that aviation should be shut down until the perfect plane and perfect air control system were worked out without learning by doing. But we got through to where we are today.
Good luck getting humans to use rationality in decision making, no matter the subject!
Your good friend Robert Wright reported that AI corrected a radiologist's issue when he had his throat cancer problem.
> It’s the same impulse that makes parents keep their children home after a school shooting on the other side of the country.
Not the point of your article, but what’s the relevance of ‘the other side of the country’? Would it be more rational if the school were nearby?
Awesome. Awesome. Awesome. So much chewy goodness. As much as there is in there to both dance with and digest, I want to treat myself to a quick point. It's curious in a way that Waymo's do not take to sidewalks. Maybe there could be exceptions in some human caused scenarios. If a problem should ever occur, it would likely be the fault of a human driver or pedestrian. I'm reminded again that the rules of the road are not a time for sociology. Expectation of expectation obviously matters. I'm also reminded of Tiger Woods, his fear maybe numbed by a opiod pain killer, expecting whatever SUV he was driving to keep up with him when even other humans can't be expected to expect him. And of course, I'm reminded of the dimensionally misaligned critiques of Freud. He made himself a household name. He's an inspiring figure. You still need Piaget broadly hiding toys to train the similarly offsettibg explanation by justified literalism.
So you're saying cardiologists are analogizing to the heart then analogizing to that, charitably speaking, to play both sides of an ev psych they don't know or have considered? It's a bit corrupt car mechanic. I analoguzed to a car yesterday later thinking I shoukd have made it Flintstone.
I recognize when I criticize DF in Boston that its not a kibbutz just as I wouldn't expect a Catholic hospital to do anything but try to hire the best they could. It's the foldibg into an abstract landscape even still knowingly against the potentially abysmal allure and colonization of digital genetics that worries me. I know changes were made at UCSF. It's a legacy of data and testing replication. I think I was meant to address the basic interpersonal occultism. It's just too stupid and I respect the institution. It's the sane problem I've had since running in SEL education at age 16. Its worse than that, though It's what UMass did to my mother coming from Catholic school. That is not easy to deal with as a violin kid practicing with her everyday. Anyway, I said one point and now I'm thinking out loud again. Thanks Paul, you're great.