The important distinction here to my mind is whether you are engaging with the AI in a task that you can think of as weightlifting or forklifting (I heard this distinction elsewhere). The basic idea is whether your main goal is to get the job done by any means necessary, or if you're engaging in some sort of deliberate practice to increase your skills or knowledge. So your argument applies mostly to things that are like forklifting...you just have to get the job done. But when dealing with students using AI to sidestep aspects of assignments that were designed to help them develop their mental muscles, then we probably need to approach it a bit differently.
As a philosopher I’m of course interested in finding out what’s true, but I am also interested in developing my own point of view, understanding what I think about certain topics, and engaging in productive conversation with others through the journals. Those things have value too. I’m not going to outsource my own thinking, even if the subcontractor is better at it than I am.
1. I agree that it's useful and important to develop an understanding of certain things, and that can't be outsourced.
2. But I'm talking about publications. Do you think there's anything wrong with getting sharp and incisive comments from other philosophers that lead you to improve the work before sending it out? If you do, then we have a real disagreement here. And I think you'd be in the minority: Many (most?) philosophical articles credit others with discussion, suggestions, and feedback. If you agree there's nothing wrong with getting such feedback, why does it matter so much if the feedback is human?
With respect to publications, I do value feedback (referee #2 excepted!), and I don’t think I have anything against AI feedback in principle. Both human and AI feedback can reach a level where you realize that you now have a co-author. When it is a human co-author there is the pleasure of jointly working out our understanding of some problem or puzzle, and we are lifting each other up. That’s lacking with AI (whom am I lifting up?). So I’m less enthused about AI co-authorship.
But suppose having an AI co-author led to a better paper. No lifting each other up, no intellectual fusion, no delightful mingling of the minds -- just a better paper, in whatever sense you think of as "better". Does this provide ANY motivation for you to go that way?
I’m honestly not sure. Think about it like this: would you like to play in a band with three robots? Suppose they are really good music-making robots. Or would you rather play music with three friends, even if they are not quite as good as the robots? With the first option at some point I’d start to wonder what my contribution is. Maybe an all-robot quartet would sound better and I can just stay home.
I’d much rather play in a band with humans than robots. More fun! But—and this is one main point of my post—having fun isn’t the primary point of scholarly or scientific work.
That is true, but there is intrinsic value to making art that is more than just having fun, and I was trying to analogize that to the intrinsic value of finding truth. I’m going to think about this some more, though. I appreciate the interaction.
Yes that’s a fair point - if I ask it to do a bibliography for me it will search in interesting & sometimes unexpected ways, but if I then ask it to do something more with that bibliography- even on the research versions - eg to give me examples, quotations, comparisons it quickly starts to confabulate very convincingly. I’m really interested by its possibilities, eg in the tedious bits of scholarly editing, but I haven’t got it to be a reliable research assistant as yet. I’ll keep pushing though & it’s interesting to read these experiences.
But what about the inaccuracies and hallucinations of AI? So far (in my experience, in my own field) it has never come up with any scholarly ideas or references I can trust, even if it sometimes synthesises ideas in interesting ways.
Well, if you find that, on balance, using AI makes your work worse, then you should definitely not use it. My own experience is very different, though. I might give it a topic or paper draft and ask it to come up with 20 relevant articles to read, 20 good examples, or 20 ideas for follow-up. Some of these will be unhelpful (and some hallucinations—though that’s less frequent these days), but almost always there is something of value here.
I've got a post coming out in the new year on why AI should be part of the peer-review process! I'm sure that will go over well among some of the academic audience.
The more that a philosopher relies on AI to write their paper, the less credit they deserve for the paper as a finished product (since the less of a causal contribution they make to it). This isn't a point unique to AI: the more your dissertation advisor helps you write your paper, the less credit you deserve. So one possible result of going down this path is that philosophers will deserve less and less credit for their publications. You can imagine that changing tenure and promotion decisions, but also various other things connected to credit. By the rationale of the post, maybe this is a price worth paying... You rack up 13 Ethics publications, and nobody gives you much credit for any of them (the credit largely goes to ChatGPT), but maybe it's worth it for the sake of producing the best papers possible and advancing truth.
One consideration, though. Suppose you get ½ credit for a paper written with AI help (or with your advisor’s help). Still, with this help, you might produce twice the number of papers in the same time, so it’s a wash …
I disagree! My view is that a virtuous epistemic agent should want to be an active producer of epistemic goods, not just a passive consumer. They should want not just to have true beliefs, but to deserve credit for the production of true beliefs, which involves making a substantial causal contribution to them. Still, journals might not care about cultivating epistemically virtuous authors. "We just want to spread true beliefs, and even if the authors are epistemic duds who deserve basically no epistemic credit for the papers we publish (because almost all the credit should got o ChatGPT), so be it!" they might declare.
Re: the analogy of using the second-best statistical test, that feels like it occupies an importantly different level of generality from “using AI”. Presumably in the stats case there’s some kind of consensus on why the best test is best and it’s likely formulated in a rather specific way (eg, mixed effects models are now preferred to an ANOVA because you can better account for multiple sources of nested variance, reduce false positives, etc).
Correspondingly we might ask more specific questions about responsible use of AI: is it irresponsible to use or not use for programming help? For writing? For hypothesis generation? And presumably the answer should depend primarily in each case on both the risks and the efficacy gains (in the same way it does with the stats example). I think that’s all consistent with what you’re arguing, it just feels relevant to me that the stats analogy involves a much more specific, targeted use case.
Re: risks, one additional counter argument could be that the increased use of AI will lead to something like “epistemic gaps” in scientific literature that will be hard to identify in any given case but which contribute, long term, to some decline in research quality or utility. But that’s totally speculative and by definition hard to falsify—basically a version of Chesterton’s fence applied to adopting this new tool.
To be clear, I think LLMs specifically are a great tool for scientific research. I use Claude regularly for programming help and I’ve also done research on using LLMs to obtain psycholinguistic norms/ratings.
Yeah this is making me think of other comparisons - is it “irresponsible” to try to publish a paper you haven’t given as a talk at a conference to get feedback on?
I rely on Chat GPT 5.2 in ethical ways I believe. I learned to navigate generative AI from Paul Bloom and Ethan Mollick - “Co- Intelligence” - plus lots of lived experience. I think I’m quite good with prompt engineering... Historically my work with gifted occupational therapists where I learned about communication and sharing feelings which I use in my prompt engineering... I ‘ve tried to tell my story so many times in the last ten years, that emotionally I need Chat GPT’s support for positive mental health. I have implemented everything in my articles in my inner systems and integration. This all represents my best effort.....
One major advantage I see with AI is they will have the chance to retain and keep knowledge faster and more reliable. There’s just a limit to how much information and data a human brain can save and retain. Assuming the knowledge is correct, this would be amazing.
At some point in school we received calculators that could perform complicated equations with multiple unknown variables, so essentially one could put in the equation, set it to solve for x and be done. So being able to actually perform the path to get to the solution became the thing we had to learn.
Next to the problem of AI reducing the need to exercise these mental muscles, the major problem I see are hallucinations. With very specific topics it’s just not particularly good at creating 100% reliable results. And one just never knows when it is wrong.
I’m seeing that in medicine where patients look up their diagnosis and treatments on ChatGTP and then refuse to believe doctors and pharmacists that tell them that these are wrong. The illusion of being an expert in a field one just is not, can be a real problem.
Yeah but those are contradictory goals. And the journals have clearly made their choice. Like, there’s not a single philosophy journal that publishes only anonymous work.
It's just Goodhart's law. When people figure out how to optimize for a measure, it ceases being a good measure. Grad students, for example, are trained to write publishable papers rather than good ones. You'd hope the two would intersect, and, at first, they do. But then people figure out how to optimize for publications, and the gap widens. That's a big part of why academia is spinning its wheels.
You wrote: “Obviously, students shouldn’t use AI to write their papers because the point of their papers isn’t to produce the best work; it’s to assess students’ ability.”
Having been a post-secondary professor, the point of their papers was to give them the experience of researching, thinking coherently, and writing; thereby developing the skills to be competent in their fields. Assessments (in my classes) and the opportunity to make corrections served to improve those skills.
You’re right — student assignments have these other goals. But these are further reasons for why students use AI (or only use them in narrow specified ways)
I agree with all this with a few quibbles or questions.
1. "Obviously, students shouldn’t use AI to write their papers because the point of their papers isn’t to produce the best work; it’s to assess students’ ability." I would not have a problem with a student using AI in the way that you suggest is acceptable for academics. They can use it it to bounce off ideas. They can ask for sources for a literature review. They can ask if there is a less awkward way of stating something. They can ask for feedback... This might depend on the assignment, and I get the distinction between student assessment and academic publication. I also think there is value in the exercise (just like the point of lifting a weight is not to get it off the ground but to exercise some muscles). Still, what we should be assessing and what skills we be useful to assess is changing with the technology.
2. I agree with your notions that editors should be credited and commentors (including generative AI) should be acknowledged. I think standards need to evolve in these areas. If you contribute an op-ed to a newspaper, and you used Claude to suggest some rewording does that need to be acknowledged? I think it would be great to do so, but that practice does not exist for humans now.
I would start by thinking about the ultimate piece of homework: the thesis. Clearly not just for assessment, but also not not for assessment. I think most papers (and indeed most homework) are mostly artefacts created for assessment purposes/to make sure that the authors are engaging in intellectual activity as well as for training to make a substantive contribution later and a small chance (!) at making a substantive contribution now. Viewed that way, restricting AI use seems not unreasonable.
(
Something that's particularly a problem in AI conferences right now is AI-generated or heavily AI-assisted papers being submitted. It's obviously frustrating to review them, because it's one thing to debug honestly constructed arguments/proofs/whatever, and another to read something that took no work to generate. This means that AI-aided papers can actually make progress slower.
(One somewhat different example: it used to be that as conference chair I could basically reject a poorly written paper because it was clearly underbaked even on a surface level; now I'm sure lots of underbaked papers get in because ChatGPT polished them)
You could say that you're talking about people who want to produce *good* papers, so those examples don't apply, since they're about bad papers -- but bad/good is a false dichotomy!
These days I try to credit the R package I used to run a particular analysis. I don’t know if there’s a clear line, but if your contribution depended on some tool/resource developed by someone else I think it makes sense to credit it. I’m not sure how tools like ChatGPT fit in here exactly or for what purposes (tightening the language in an abstract vs generating hypotheses).
On the other hand, I don’t credit Donald Knuth and LaTeX for everything I write, and most people don’t have acknowledgments for the Microsoft corporation in their work.
That's a good point. I wonder whether there's some set of fuzzy descriptive criteria one could identify here that people seem to follow within a given field—I assume there must be someone working on this kind of question (maybe in science and technology studies?).
The important distinction here to my mind is whether you are engaging with the AI in a task that you can think of as weightlifting or forklifting (I heard this distinction elsewhere). The basic idea is whether your main goal is to get the job done by any means necessary, or if you're engaging in some sort of deliberate practice to increase your skills or knowledge. So your argument applies mostly to things that are like forklifting...you just have to get the job done. But when dealing with students using AI to sidestep aspects of assignments that were designed to help them develop their mental muscles, then we probably need to approach it a bit differently.
As a philosopher I’m of course interested in finding out what’s true, but I am also interested in developing my own point of view, understanding what I think about certain topics, and engaging in productive conversation with others through the journals. Those things have value too. I’m not going to outsource my own thinking, even if the subcontractor is better at it than I am.
I like this, but two ideas are blurred together.
1. I agree that it's useful and important to develop an understanding of certain things, and that can't be outsourced.
2. But I'm talking about publications. Do you think there's anything wrong with getting sharp and incisive comments from other philosophers that lead you to improve the work before sending it out? If you do, then we have a real disagreement here. And I think you'd be in the minority: Many (most?) philosophical articles credit others with discussion, suggestions, and feedback. If you agree there's nothing wrong with getting such feedback, why does it matter so much if the feedback is human?
With respect to publications, I do value feedback (referee #2 excepted!), and I don’t think I have anything against AI feedback in principle. Both human and AI feedback can reach a level where you realize that you now have a co-author. When it is a human co-author there is the pleasure of jointly working out our understanding of some problem or puzzle, and we are lifting each other up. That’s lacking with AI (whom am I lifting up?). So I’m less enthused about AI co-authorship.
But suppose having an AI co-author led to a better paper. No lifting each other up, no intellectual fusion, no delightful mingling of the minds -- just a better paper, in whatever sense you think of as "better". Does this provide ANY motivation for you to go that way?
I’m honestly not sure. Think about it like this: would you like to play in a band with three robots? Suppose they are really good music-making robots. Or would you rather play music with three friends, even if they are not quite as good as the robots? With the first option at some point I’d start to wonder what my contribution is. Maybe an all-robot quartet would sound better and I can just stay home.
I’d much rather play in a band with humans than robots. More fun! But—and this is one main point of my post—having fun isn’t the primary point of scholarly or scientific work.
That is true, but there is intrinsic value to making art that is more than just having fun, and I was trying to analogize that to the intrinsic value of finding truth. I’m going to think about this some more, though. I appreciate the interaction.
Yes that’s a fair point - if I ask it to do a bibliography for me it will search in interesting & sometimes unexpected ways, but if I then ask it to do something more with that bibliography- even on the research versions - eg to give me examples, quotations, comparisons it quickly starts to confabulate very convincingly. I’m really interested by its possibilities, eg in the tedious bits of scholarly editing, but I haven’t got it to be a reliable research assistant as yet. I’ll keep pushing though & it’s interesting to read these experiences.
But what about the inaccuracies and hallucinations of AI? So far (in my experience, in my own field) it has never come up with any scholarly ideas or references I can trust, even if it sometimes synthesises ideas in interesting ways.
Well, if you find that, on balance, using AI makes your work worse, then you should definitely not use it. My own experience is very different, though. I might give it a topic or paper draft and ask it to come up with 20 relevant articles to read, 20 good examples, or 20 ideas for follow-up. Some of these will be unhelpful (and some hallucinations—though that’s less frequent these days), but almost always there is something of value here.
I've got a post coming out in the new year on why AI should be part of the peer-review process! I'm sure that will go over well among some of the academic audience.
I love this post!
The more that a philosopher relies on AI to write their paper, the less credit they deserve for the paper as a finished product (since the less of a causal contribution they make to it). This isn't a point unique to AI: the more your dissertation advisor helps you write your paper, the less credit you deserve. So one possible result of going down this path is that philosophers will deserve less and less credit for their publications. You can imagine that changing tenure and promotion decisions, but also various other things connected to credit. By the rationale of the post, maybe this is a price worth paying... You rack up 13 Ethics publications, and nobody gives you much credit for any of them (the credit largely goes to ChatGPT), but maybe it's worth it for the sake of producing the best papers possible and advancing truth.
I agree with all of that.
One consideration, though. Suppose you get ½ credit for a paper written with AI help (or with your advisor’s help). Still, with this help, you might produce twice the number of papers in the same time, so it’s a wash …
Credit may be what people who want a job care about, but it’s not the point of the endeavor!
I disagree! My view is that a virtuous epistemic agent should want to be an active producer of epistemic goods, not just a passive consumer. They should want not just to have true beliefs, but to deserve credit for the production of true beliefs, which involves making a substantial causal contribution to them. Still, journals might not care about cultivating epistemically virtuous authors. "We just want to spread true beliefs, and even if the authors are epistemic duds who deserve basically no epistemic credit for the papers we publish (because almost all the credit should got o ChatGPT), so be it!" they might declare.
These are good points.
Re: the analogy of using the second-best statistical test, that feels like it occupies an importantly different level of generality from “using AI”. Presumably in the stats case there’s some kind of consensus on why the best test is best and it’s likely formulated in a rather specific way (eg, mixed effects models are now preferred to an ANOVA because you can better account for multiple sources of nested variance, reduce false positives, etc).
Correspondingly we might ask more specific questions about responsible use of AI: is it irresponsible to use or not use for programming help? For writing? For hypothesis generation? And presumably the answer should depend primarily in each case on both the risks and the efficacy gains (in the same way it does with the stats example). I think that’s all consistent with what you’re arguing, it just feels relevant to me that the stats analogy involves a much more specific, targeted use case.
Re: risks, one additional counter argument could be that the increased use of AI will lead to something like “epistemic gaps” in scientific literature that will be hard to identify in any given case but which contribute, long term, to some decline in research quality or utility. But that’s totally speculative and by definition hard to falsify—basically a version of Chesterton’s fence applied to adopting this new tool.
To be clear, I think LLMs specifically are a great tool for scientific research. I use Claude regularly for programming help and I’ve also done research on using LLMs to obtain psycholinguistic norms/ratings.
Yeah this is making me think of other comparisons - is it “irresponsible” to try to publish a paper you haven’t given as a talk at a conference to get feedback on?
I rely on Chat GPT 5.2 in ethical ways I believe. I learned to navigate generative AI from Paul Bloom and Ethan Mollick - “Co- Intelligence” - plus lots of lived experience. I think I’m quite good with prompt engineering... Historically my work with gifted occupational therapists where I learned about communication and sharing feelings which I use in my prompt engineering... I ‘ve tried to tell my story so many times in the last ten years, that emotionally I need Chat GPT’s support for positive mental health. I have implemented everything in my articles in my inner systems and integration. This all represents my best effort.....
One major advantage I see with AI is they will have the chance to retain and keep knowledge faster and more reliable. There’s just a limit to how much information and data a human brain can save and retain. Assuming the knowledge is correct, this would be amazing.
At some point in school we received calculators that could perform complicated equations with multiple unknown variables, so essentially one could put in the equation, set it to solve for x and be done. So being able to actually perform the path to get to the solution became the thing we had to learn.
Next to the problem of AI reducing the need to exercise these mental muscles, the major problem I see are hallucinations. With very specific topics it’s just not particularly good at creating 100% reliable results. And one just never knows when it is wrong.
I’m seeing that in medicine where patients look up their diagnosis and treatments on ChatGTP and then refuse to believe doctors and pharmacists that tell them that these are wrong. The illusion of being an expert in a field one just is not, can be a real problem.
Academic journals exist mainly to certify people and not ideas. If they wanted to certify ideas, they'd make all publications anonymous.
well, they serve multiple goals.
Yeah but those are contradictory goals. And the journals have clearly made their choice. Like, there’s not a single philosophy journal that publishes only anonymous work.
Why are they contradictory? When Einstein published his work, his name was associated with it AND the ideas got conveyed to the world.
Human nature being what it is, I think giving people credit for their ideas is an excellent way of encouraging the generation of ideas.
Human nature being what it is, the incentivized pursuit of knowledge will be crowded out by the pursuit of its measure.
Huh. Interesting. I'm not sure why you think this, though.
It's just Goodhart's law. When people figure out how to optimize for a measure, it ceases being a good measure. Grad students, for example, are trained to write publishable papers rather than good ones. You'd hope the two would intersect, and, at first, they do. But then people figure out how to optimize for publications, and the gap widens. That's a big part of why academia is spinning its wheels.
You wrote: “Obviously, students shouldn’t use AI to write their papers because the point of their papers isn’t to produce the best work; it’s to assess students’ ability.”
Having been a post-secondary professor, the point of their papers was to give them the experience of researching, thinking coherently, and writing; thereby developing the skills to be competent in their fields. Assessments (in my classes) and the opportunity to make corrections served to improve those skills.
You’re right — student assignments have these other goals. But these are further reasons for why students use AI (or only use them in narrow specified ways)
I agree with all this with a few quibbles or questions.
1. "Obviously, students shouldn’t use AI to write their papers because the point of their papers isn’t to produce the best work; it’s to assess students’ ability." I would not have a problem with a student using AI in the way that you suggest is acceptable for academics. They can use it it to bounce off ideas. They can ask for sources for a literature review. They can ask if there is a less awkward way of stating something. They can ask for feedback... This might depend on the assignment, and I get the distinction between student assessment and academic publication. I also think there is value in the exercise (just like the point of lifting a weight is not to get it off the ground but to exercise some muscles). Still, what we should be assessing and what skills we be useful to assess is changing with the technology.
2. I agree with your notions that editors should be credited and commentors (including generative AI) should be acknowledged. I think standards need to evolve in these areas. If you contribute an op-ed to a newspaper, and you used Claude to suggest some rewording does that need to be acknowledged? I think it would be great to do so, but that practice does not exist for humans now.
I would start by thinking about the ultimate piece of homework: the thesis. Clearly not just for assessment, but also not not for assessment. I think most papers (and indeed most homework) are mostly artefacts created for assessment purposes/to make sure that the authors are engaging in intellectual activity as well as for training to make a substantive contribution later and a small chance (!) at making a substantive contribution now. Viewed that way, restricting AI use seems not unreasonable.
(
Something that's particularly a problem in AI conferences right now is AI-generated or heavily AI-assisted papers being submitted. It's obviously frustrating to review them, because it's one thing to debug honestly constructed arguments/proofs/whatever, and another to read something that took no work to generate. This means that AI-aided papers can actually make progress slower.
(One somewhat different example: it used to be that as conference chair I could basically reject a poorly written paper because it was clearly underbaked even on a surface level; now I'm sure lots of underbaked papers get in because ChatGPT polished them)
You could say that you're talking about people who want to produce *good* papers, so those examples don't apply, since they're about bad papers -- but bad/good is a false dichotomy!
)
The people in the comments all seem to assume you would be doing something fraudulent with AI.
acknowledge the help of the AI—it’s dishonest to take credit for work that’s not your own
I wonder -- I don't credit a thesaurus or calculator -- what, specifically, is different here (unless directly quoting)?
These days I try to credit the R package I used to run a particular analysis. I don’t know if there’s a clear line, but if your contribution depended on some tool/resource developed by someone else I think it makes sense to credit it. I’m not sure how tools like ChatGPT fit in here exactly or for what purposes (tightening the language in an abstract vs generating hypotheses).
On the other hand, I don’t credit Donald Knuth and LaTeX for everything I write, and most people don’t have acknowledgments for the Microsoft corporation in their work.
That's a good point. I wonder whether there's some set of fuzzy descriptive criteria one could identify here that people seem to follow within a given field—I assume there must be someone working on this kind of question (maybe in science and technology studies?).
Is it irresponsible to use AI to summarize / normal-language-ize academic papers?
;-)