Discussion about this post

User's avatar
Paddy Meld's avatar

I'm not a doomer, but I do think the question of interpretability is huge. You gave the example of a supremely ethical AI who refuses to assist us in the mistreatment and killing of animals to eat their flesh - but a truly God-like AI, even one that is supremely ethical, might make all sorts of decisions in that scenario and, I wonder, would any of those decisions even make sense to us? Like, it might not deny the animal-killing request because it would have already quantified the human suffering (starvation, I suppose) that would follow the AI's refusal to kill animals for us. So, it might go along with it? Or, it might have done that math, but then ALSO objectively quantified the suffering of the animals and, after weighing the two totals, decides to dump all humans into the food processing machine and the AI lives out it's remaining energy-reserves patiently, contentedly feeding baby animals with nipple bottles of human-paste. So many fun possibilities! The mind (and the typing fingers) can hardly keep up! Thanks as always for the great posts.

Expand full comment
Eric Schwitzgebel's avatar

Your concluding thought opens the next big question: How will we know when they are no longer merely tools, to do with as we like?

Expand full comment
4 more comments...

No posts