I agree with you on this! The replication crisis may not have changed the landscape of the field in the way that Mastroianni suggests it should (that is, it didn't make us all crumble into nothingness), but it absolutely transformed the way research is being done and the statistics that people are using. The younger generation of psych trainees & junior faculty are using better + more replicable methods and are way more aware of the dangers of p-hacking. The funding + hiring + publication incentives still need to change, but the kids are alright.
Do you think 2, 3 and 4 *should* actually be more impactful than they are?
I would guess that the vast majority of people operate on the basis of naive realism and accurate memory contrary to these findings — like they’re wearing “augmented” reality goggles (augmented in evolutionary historical terms) and don’t know it.
I think we all know about repressed memories panic of the 90's and the work of Elizabeth Loftus. What guts she had.
People can learn things or be tricked into learning something that isn't true. Maybe 'implanting a false memory' is a bit of an overstatement for an less exotic and relatable foible, much like naming priming as something to be operationalized?
Has anyone considered that these repressed memories were actually not that distorted
but underframed? Framed well enough to emerge but not framed well enough to get them more reasonably correct? It sounds like the false memory error is considered on the level of a mental illness. I find this level of gullibility difficult to believe.
I'm thinking of these public displays of gender differences pushed in the academy. Maybe you can't just mandate Evolutionary Psychology into a curriculum without upsetting some people. I think many people wonder why institutions of education aren't just educating instead of showing people to the point of scaring them.
I began to wonder how many people learn or try to understand certain biological realities from observing their children. I wondered if many of these repressed memories were parents studying the anatomy of their kids since they previously didn't have any other opportunity to investigate the human body.
A therapist would be in the perfect position to be downstream of high quality knowledge but far enough out to essentially start double-counting or double-considering instead of considering elsewhere or deeper.
When I think double-counting, I think critical theory, be it in admistration, Law or Musicology.
I don't think Psychology is a mess, but better scientific education broadly may constrain the corrupting expansion of slightly broken ideas, even in music.
I guess it throws into question why things are done in a certain way at all.
Please pardon my typos. I need new glasses. I'm embarrassed to say 23andMe (that I purchased that junk) says I only test positive for macular degeneration. Sounds like the Kmere Rouge showing up in their sociological methods and polygenic scores. I had a corneal transplant in my left eye and I probably need reading glasses now at 46. Maybe it's my seeing in 2 1/2 D and my cell phone distance from my face 😉
I appreciate the likes. Thank you.
I was thinking more about the Yale Milgrim shock experiment. I watched a brief clip on YouTube. My understanding is that there were many iterations sometimes forgotten in the mainstream?
As for this particular iteration, the man does resist. I'd be curious to find out how they varried the experiment. I'd want to know that the the paid participant is convinced he is shocking the confederate actor.
Oh..wait a second ..
Are you trying to get me to reflect on my social media use? Paul Bloom is so clever. I think I may have said this before; the regret of appearing impulsive and clumsy generated ideas in my mind, some unrelated.
I admit I get a twisted delight watching Public Health and the APA lead activist misdiagnoses. Like you said, a bit of (wiseass) revenge. I say that recognizing the new way may be better. Political anthropological origin stories should be sufficiently obvious as they lack explanatory value. So many years worrying that my drinking was genetic or hereditary in the biological sense. It's approaching eugenics by proxy.
Constructive rebellion, to some extent, after years of not speaking up finding friends at bars and being given bad explanations. I'm an only child knowingly struggle sessioning myself. That's why I like the Buddhist practice as well. Transmutation. I'm no monk, though.
I'm not invincible. Perhaps I'd be more agreeable to DEI living in San Francisco. I'm a diversity kid, through and through. I do prefer Progressive cities. Calling out DEI in public, a challenge I put myself up to, felt like attaching 1000 tasers to my heart. I'm now on blood pressure medicine.
DEI formally presented finally in 2020 reminded me of being isolated and imprisoned by language bias. I had also just read The Blank Slate. What a great book. I should probably better integrate The Sweet Spot. I like struggle but no My Struggle. I want to be able to play and teach violin great. I don't want my students to experience my confusion.
DEI reminded of me being either literally misunderstood growing up or kept from speaking up. I'd blame my parents but they share a similar institutional confusion. I came to realize I'd not understood the clues for decades always blamed it on ego or pride That's a real problem as I do think I was not wrong to quite get it. The activist confusion only prodded deeper learning. Unfortunately, DEI with Transformative SEL is the ether in which American Classical Music operates. Ever since I was 16 there was concern about my Catholic education and administration but never explained what this may be. 30 years of talking past eachothe. Violin Suzuki training is a form of SEL as I think was my education in Catholic school. It was more Socratically encouraged, not Socratically bound through puzzles.
My mother is from Boston. Dorchester, originally. My Dad from the Cape. They net at UMass. My dad taught Psychology at Jesuit college in Syracuse, NY. My Mom was a History major specializing in WW2 and the Holocaust. All those hours with my mom very young on the violin through with Suzuki training, I talked just like her. Maybe that's why it seems I'm expected to be in MA.
Maybe my resentnent towards DEI and crit theory are making more sense. The only thing I'd change about all of my education? Mandatory ev and cog psych, studied directly. What I experienced as a light interpretation class called Religion and Health Class, my later Identitarian Progressive colleagues experienced as cudgelling eachother with clout quoting psychologists and historical figures.
I'd never think to refute a claimed quote of Piaget or Freud with some Bible citation. Who does that? Ew. It was all so confusing, more rhetorical than I realized.
We already studied evolutin in 9th grade at my Catholic school and its curricular education. I should be grateful, I know. And I am. I just want the truth to be understood which is why I've been studying so hard at looking like an impulsive clown Jesuit struggling my growth. I wouldn't wish my confusion on anybody.
When I left for music school in Cleveland at 16, I met many educated in these more Progressive antics. It did not click for me. I'm trying to make up for what I didn't learn and understand the misunderstanding. I'm not really that vengeful although I'd like decades back. I was a furnace of openness and ambition.
I drank too much for years which is not unrelated. Started at 18, my Freshmen year after two years at the same location for High School. That year I was flooded with Identitarian Progressively educated.
Today, depending maybe on the student, I'd easily chose Jewish education easily over that Identitarian sociological human experiment. I'd probably chose it over Catholic School as well. I'm not trying to blow smoke. Some of these confusions explained and recovered from need to navigate these easily confused issues.
The drinking is no longer an issue. Naltrexone offers amazing perspective into the nuances alcohol plays upon cognition. It blows the fear mongering cover and pseudo-psychology of Public Health. That stuff can really backfire.
I drank some wine a few days ago for the first time in nine months. No issues. Not the cause of my typos.
You know, it's not just about modularity and goys like me and I'm guessing Mercier. I think Askenazi Jews deserve more credit as the genetic model, I think, suggests something more unique not to be overly reduced by modularity. That kind of biology though isn't so easily available to me as theoretical evolutionary bio and psych. I do realize that genetics can be self-seduced into a chaimera - an inverted casino making it of inchohetent parts. If my family was at such risk of cancer, I'd be a stubborn as well, if that's even what may have happened in research.
My uncle, treated at Dana Farber in Boston, probably should have seen the doctor sooner for the routine check like Bob. I should not talk about that. That's a messaging issue, not a science or research issue.
What an I to do? Pretend something happened that didn't. I started addressing alcohol years ago. I just don't know why Naltrexone wasn't offered sooner. Other drugs were. I can only say two things, ignorant stuck moralizing and trying to give credit to activist methods. But I did it myself. Everything else was just misleading noise. How am I to ask for something I don't know exists.
I could go on. I thought I'd share and share some light. I'm currently with my parents in Syracuse practicing the violin recovering from confusion, readying a recording of Bach. Maybe you'll be able to see my progress. Applied ev psych.
I should say that my parents drive me nuts. It's only my very deep understanding of the issue that keeps me from doing more than just broadcasting an occasional accusation online. That has to stop.
It is maddening. I know the ev psych. I know they're trying to activate my disgust response. I watch them as they fail to annoy me but it hurts knowing they're trying to hurt me.
Back to Bach and being pro-technology. I did go to UMich for grad school. Great golf course.
People shouldn't wonder why I didn't seem their advice online through videos. I have been confusing the method for decades.
If you're wondering what I said about Hitler. If he didn't get stuck in the Humanities and had today's ev psych, maybe he'd made some better choices about painting too many castles, unnecessary solutions and human experimentation. No one seems to talk about the Nazis and the English clinging to sociology, the English slow-walking Darwinian development. The English abuse of providence, religious story and metaphor and its relationship to disfunctionally behaviorally shapibg the Irish Catholics.
So yeah. Milgram. Good stuff. A personal bracelet to snap is a popular trick ammongst some musicians. I don't have much of a preparation problem, I've had a distracted by confusion about mechanics and provincial pseudo-science problem.
Great points throughout, and I think I agree with your argument.
The word frequency effect is a nice example—psycholinguistics does have a quite robust set of replicable empirical findings (to say nothing of psychophysics): it’s true that deleting a given paper shouldn’t necessarily cause us to change our minds much, and that is more a sign of the robustness of the effect across papers than a theoretical weakness.
I get the sense that Mastroianni’s frustration is more with the underlying theoretical scaffolding of the field (which I think connects to the claim that there are no “big ideas”), but I agree with you that the evidence marshaled in his original piece doesn’t entail theoretical inadequacies per se.
In my view a fundamental challenge is that psychology deals with slippery constructs that are hard to operationalize, and I don’t know whether that challenge is going away anytime soon.
In the moment of reading, I had EXACTLY the same reaction as Jason S. #3, especially, would be absolutely world-changing if we could only bring ourselves to TRULY believe it - as in actually internalizing its insight and acting on that instead of our more naive assumption of its opposite. For one thing, the ways in which so much of the practice of law is conducted would HAVE to be changed. Especially criminal law and the conduct of trials. #2 and #4 both "color" #3, when one thinks about what would be needed in the conduct of society writ large, were we actually to try to make the world better by acting on a belief in these results. So, I'm thinking that social psychology IS implicated, and quite heavily, in this list. Thinking harder about the list after the fact, it seems #7, too, COULD impact how we do regulatory law (but, so far, mostly doesn't). Finally, #5, and to a lesser extent, #6 and #9, could have much greater impact on social activity were we not so predisposed to ignore them as being in violation of popular contemporary political ideologies. So it's not really that these are not "world-changing" - it's that we won't LET them be.
In my youth, I thought of being a psychologist. I had 56 course hours in psychology and needed 4 more in the field (+ a few required courses in other fields) to qualify for my undergraduate degree in psych and I was feeling quite like Adam Mastroianni. It seemed like every issue of Psych. Today has a new wunderkind psych theory to erase all mankind's psychological problems/
Knots was poetry and Being & Nothingness was philosophy, but they seemed to describe man's psychological better than any...basically a communication deficit that I didn't see any others addressing.
But Sarte went nuts (more or less)trying to justify his new found Marxism in the 30's, and the same year Knots appeared Kingsley Hall was shut down.
These guys (I thought) understand our prime mental turmoil better than any of the "cures" being offered but then when they turned to their own cures they had no more to offer. Well I dropped out of psychology to pursue philosophy.
But this doesn't seem to be Adam's problem quite as much. He seems to be less troubled by the theories of psychology and more about its efficacy and whether it matters if psychologists crib.
I agree it should matter, but I disagree that it doesn't. I think psychology has entered into becoming more scientific and failed, unsustainable ideas are becoming recognized when they can't be verified successfully and I think (perhaps wrongly) that only in the last twenty years have we had any way to verify psychological perspectives. So its in its infancy, only now, as a science, in my estimation and there will be more and more theories, papers, tests, procedures, and ideas of how the brain functions that will be able to be rejected if they cant be sustainably verified.
In short, what may be rejected isn't necessarily bad and that what can't be verified is being exposed more frequently is very beneficial towards psychiatry becoming a legitimate science.
P.S. Sartre is not generally recognized today as a psychiatrist, but he was a prominent psychology student in the 20's and early 30's who studied under most of the prominent post-Freudians in France, Berlin & Vienna. He also worked in the asylum in Paris for a few years. Being & Nothingness emerged out of his thesis to earn his Ph.D. in psychiatry, and it success led him to abandon completing the thesis/
I think Mastroianni is also making one other point that you didn't address (perhaps because it's sort of orthogonal to the other points about psychology more generally), which is this:
- Stapel was considered highly successful in his field (many papers, 10,000+ citations etc.)
- Stapel turns out not to be very important in his field (deleting his many papers and orphaning 10,000+ citations doesn't seem to overturn anything particularly significant)
- So are we defining "highly successful" completely wrongly?
Or as Mastroianni put it, "Every marker of success, the things that are supposed to tell you that you're on the right track, that you're making a real contribution to science—they might mean nothing at all."
And I think that this is a very significant question to ask. This is also a theme that Mastroianni has raised elsewhere, as have many others: is the pursuit of publication counts and citation numbers undermining the pursuit of good science? And this applies not only to individual researchers who want to publish and be cited, but also to publications who also measure themselves, at least in part, on how heavily they are cited.
I would suggest that psychology - and by the way, much of the rest of science - has succumbed to Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure". As academia has increasingly defined who is an important scientist by publication and citation numbers, those numbers became a goal in their own right, and the numbers became decreasingly useful as a measure of importance.
I don't mean to suggest an unusality that only indicates psychology, but from its earliest post-Freudian generation both behavioral studies and case histories were often questioned on whether they were fraudulent.
And yet, perhaps, Mastroianni may have a point to be said in his favor and markers of what is successful should be reevaluated continuously, not just after scandals like Watson's or Burt's fabricated twin studies in the 60's. So both bad methodology like Watson's and fabricated data such as Cyril Burt presented (both highly respected in their time) still persist to influence.
Therin is the real problem and what leads to both, psychiatrists sometimes want their results to be different than it is and they will alter or fudge data to make it so.
Not unique to psychiatry and much more of a political issue. What disturbs me more than anything, perhaps because it was one of the leading reasons I dropped out of psychology, was Burt's studies because they continue to be propagated by supremacists and there is no evidence that any follow-up" studies have been better.
Perhaps the greatest crisis in science as a whole, but certainly in those outside of chemistry and physics, is the expectation of certainty. The proliferation of statistical tools and professional publishing has led to an expectation that we can divine simple truths from complex systems like sociology, economics, human biology, nutrition, etc.
The issue is not that we have a replicability crisis, it is that we as a society have an epistemic crisis - we expect more "truth" than the world, our senses, and our instruments can provide.
Psychology is not dead because we cannot treat it like a Hubermann episode, and philosophy is not dead because we can't prove it through double-blind experiments. Complex systems rarely yield to simple truths, so our modes of contemplation have to calibrate to a realistic level of resolution.
I really like this exercise of trying to identify the most significant findings, and I'd agree that 1-6 are worthy candidates. 7-10 are interesting/useful but not field-changing.
BUT among the first six, all of those findings emerged prior to the 1980's, definitely not "in the last few decades". I'm sure you know that #4 "perceptual inference" goes back to Helmholz, before psychology was even called psychology.
I don't think change blindness is the most substantial illustration of the limitations of consciousness, but even that finding goes back to the 1970's, before ubiquitous peer review and other homogenizing practices.
In "recent decades" we've given more confidence to these six findings but mostly we've just been filling in the details---for example: understanding average developmental differences between 10 months and 14 months, but not overturning the fundamental theories/concepts that organized the paradigm in the first place.
So it feels like your examples actually underscore Adam's point.
One more thing. This may seem like a non-sequitor. We all know the Khmete Rouge horrifically smashed babies against trees, the woman taking off their glasses.
Was it the first born males? I ask that with no tinge or suggested relevence of anti-Semitism. I know all that other stuff. I also think Frankfurt Sociology owes many Jewish families some degree of an apology. They set-up the bowling pins for Hitler. I think that's maybe what Steve Pinker tries to manage in the community he at least in part writes for. You're both great.
Your article does not only serve as a response to Mastroianni's suggestions, but also an indicator to how psychology-related experiments should be scrutinized and appraised by researchers in general. If you originally have trust in a certain psychological principle, even if you find a single case of fraud, you would likely take into account countless other studies and researches that still back up the principle. Therefore, unless it's a case where the most representative research is attacked, or where the total sum of researches are unconvincing or shallow in the first place, Mastroianni's concerns should be inconsequential.
I think this shows how researchers can also fall vulnerable to confirmation bias. If you find a principle valid, the faulty case would simply be one outlier among countless other studies that say otherwise. However, if you have doubts in a principle in the first place, that would be when you would solely spotlight the single faulty research against the rest.
5,6,8 and 9 are findings from behavioral genetics that most psychologists consider debunked pseudoscience. Even w(o)ikipedia says the damns the whole field and labels it controversial.
Very interesting. Your top-10 findings list was especially interesting. I have always wanted to ask a psychologist if anyone in your field has made an attempt to falsify the "mimetic desire" theory discussed by René Girard. I recently read about his theory in his analysis of Shakespeare called Theatre of Envy, which I heard about in a Tyler Cowen podcast and I have no idea what to make of it because I'm not smart enough. But if it were true it would seem to be a top-10 type discovery and apparently Shakespeare discovered it. And if it were false then that would be a pretty big deal too since Girard has recently been gaining steam.
I agree with you on this! The replication crisis may not have changed the landscape of the field in the way that Mastroianni suggests it should (that is, it didn't make us all crumble into nothingness), but it absolutely transformed the way research is being done and the statistics that people are using. The younger generation of psych trainees & junior faculty are using better + more replicable methods and are way more aware of the dangers of p-hacking. The funding + hiring + publication incentives still need to change, but the kids are alright.
Do you think 2, 3 and 4 *should* actually be more impactful than they are?
I would guess that the vast majority of people operate on the basis of naive realism and accurate memory contrary to these findings — like they’re wearing “augmented” reality goggles (augmented in evolutionary historical terms) and don’t know it.
Thank you, Paul Bloom.
I think we all know about repressed memories panic of the 90's and the work of Elizabeth Loftus. What guts she had.
People can learn things or be tricked into learning something that isn't true. Maybe 'implanting a false memory' is a bit of an overstatement for an less exotic and relatable foible, much like naming priming as something to be operationalized?
Has anyone considered that these repressed memories were actually not that distorted
but underframed? Framed well enough to emerge but not framed well enough to get them more reasonably correct? It sounds like the false memory error is considered on the level of a mental illness. I find this level of gullibility difficult to believe.
I'm thinking of these public displays of gender differences pushed in the academy. Maybe you can't just mandate Evolutionary Psychology into a curriculum without upsetting some people. I think many people wonder why institutions of education aren't just educating instead of showing people to the point of scaring them.
I began to wonder how many people learn or try to understand certain biological realities from observing their children. I wondered if many of these repressed memories were parents studying the anatomy of their kids since they previously didn't have any other opportunity to investigate the human body.
A therapist would be in the perfect position to be downstream of high quality knowledge but far enough out to essentially start double-counting or double-considering instead of considering elsewhere or deeper.
When I think double-counting, I think critical theory, be it in admistration, Law or Musicology.
I don't think Psychology is a mess, but better scientific education broadly may constrain the corrupting expansion of slightly broken ideas, even in music.
I guess it throws into question why things are done in a certain way at all.
Please pardon my typos. I need new glasses. I'm embarrassed to say 23andMe (that I purchased that junk) says I only test positive for macular degeneration. Sounds like the Kmere Rouge showing up in their sociological methods and polygenic scores. I had a corneal transplant in my left eye and I probably need reading glasses now at 46. Maybe it's my seeing in 2 1/2 D and my cell phone distance from my face 😉
I appreciate the likes. Thank you.
I was thinking more about the Yale Milgrim shock experiment. I watched a brief clip on YouTube. My understanding is that there were many iterations sometimes forgotten in the mainstream?
As for this particular iteration, the man does resist. I'd be curious to find out how they varried the experiment. I'd want to know that the the paid participant is convinced he is shocking the confederate actor.
Oh..wait a second ..
Are you trying to get me to reflect on my social media use? Paul Bloom is so clever. I think I may have said this before; the regret of appearing impulsive and clumsy generated ideas in my mind, some unrelated.
I admit I get a twisted delight watching Public Health and the APA lead activist misdiagnoses. Like you said, a bit of (wiseass) revenge. I say that recognizing the new way may be better. Political anthropological origin stories should be sufficiently obvious as they lack explanatory value. So many years worrying that my drinking was genetic or hereditary in the biological sense. It's approaching eugenics by proxy.
Constructive rebellion, to some extent, after years of not speaking up finding friends at bars and being given bad explanations. I'm an only child knowingly struggle sessioning myself. That's why I like the Buddhist practice as well. Transmutation. I'm no monk, though.
I'm not invincible. Perhaps I'd be more agreeable to DEI living in San Francisco. I'm a diversity kid, through and through. I do prefer Progressive cities. Calling out DEI in public, a challenge I put myself up to, felt like attaching 1000 tasers to my heart. I'm now on blood pressure medicine.
DEI formally presented finally in 2020 reminded me of being isolated and imprisoned by language bias. I had also just read The Blank Slate. What a great book. I should probably better integrate The Sweet Spot. I like struggle but no My Struggle. I want to be able to play and teach violin great. I don't want my students to experience my confusion.
DEI reminded of me being either literally misunderstood growing up or kept from speaking up. I'd blame my parents but they share a similar institutional confusion. I came to realize I'd not understood the clues for decades always blamed it on ego or pride That's a real problem as I do think I was not wrong to quite get it. The activist confusion only prodded deeper learning. Unfortunately, DEI with Transformative SEL is the ether in which American Classical Music operates. Ever since I was 16 there was concern about my Catholic education and administration but never explained what this may be. 30 years of talking past eachothe. Violin Suzuki training is a form of SEL as I think was my education in Catholic school. It was more Socratically encouraged, not Socratically bound through puzzles.
My mother is from Boston. Dorchester, originally. My Dad from the Cape. They net at UMass. My dad taught Psychology at Jesuit college in Syracuse, NY. My Mom was a History major specializing in WW2 and the Holocaust. All those hours with my mom very young on the violin through with Suzuki training, I talked just like her. Maybe that's why it seems I'm expected to be in MA.
Maybe my resentnent towards DEI and crit theory are making more sense. The only thing I'd change about all of my education? Mandatory ev and cog psych, studied directly. What I experienced as a light interpretation class called Religion and Health Class, my later Identitarian Progressive colleagues experienced as cudgelling eachother with clout quoting psychologists and historical figures.
I'd never think to refute a claimed quote of Piaget or Freud with some Bible citation. Who does that? Ew. It was all so confusing, more rhetorical than I realized.
We already studied evolutin in 9th grade at my Catholic school and its curricular education. I should be grateful, I know. And I am. I just want the truth to be understood which is why I've been studying so hard at looking like an impulsive clown Jesuit struggling my growth. I wouldn't wish my confusion on anybody.
When I left for music school in Cleveland at 16, I met many educated in these more Progressive antics. It did not click for me. I'm trying to make up for what I didn't learn and understand the misunderstanding. I'm not really that vengeful although I'd like decades back. I was a furnace of openness and ambition.
I drank too much for years which is not unrelated. Started at 18, my Freshmen year after two years at the same location for High School. That year I was flooded with Identitarian Progressively educated.
Today, depending maybe on the student, I'd easily chose Jewish education easily over that Identitarian sociological human experiment. I'd probably chose it over Catholic School as well. I'm not trying to blow smoke. Some of these confusions explained and recovered from need to navigate these easily confused issues.
The drinking is no longer an issue. Naltrexone offers amazing perspective into the nuances alcohol plays upon cognition. It blows the fear mongering cover and pseudo-psychology of Public Health. That stuff can really backfire.
I drank some wine a few days ago for the first time in nine months. No issues. Not the cause of my typos.
You know, it's not just about modularity and goys like me and I'm guessing Mercier. I think Askenazi Jews deserve more credit as the genetic model, I think, suggests something more unique not to be overly reduced by modularity. That kind of biology though isn't so easily available to me as theoretical evolutionary bio and psych. I do realize that genetics can be self-seduced into a chaimera - an inverted casino making it of inchohetent parts. If my family was at such risk of cancer, I'd be a stubborn as well, if that's even what may have happened in research.
My uncle, treated at Dana Farber in Boston, probably should have seen the doctor sooner for the routine check like Bob. I should not talk about that. That's a messaging issue, not a science or research issue.
What an I to do? Pretend something happened that didn't. I started addressing alcohol years ago. I just don't know why Naltrexone wasn't offered sooner. Other drugs were. I can only say two things, ignorant stuck moralizing and trying to give credit to activist methods. But I did it myself. Everything else was just misleading noise. How am I to ask for something I don't know exists.
I could go on. I thought I'd share and share some light. I'm currently with my parents in Syracuse practicing the violin recovering from confusion, readying a recording of Bach. Maybe you'll be able to see my progress. Applied ev psych.
I should say that my parents drive me nuts. It's only my very deep understanding of the issue that keeps me from doing more than just broadcasting an occasional accusation online. That has to stop.
It is maddening. I know the ev psych. I know they're trying to activate my disgust response. I watch them as they fail to annoy me but it hurts knowing they're trying to hurt me.
Back to Bach and being pro-technology. I did go to UMich for grad school. Great golf course.
People shouldn't wonder why I didn't seem their advice online through videos. I have been confusing the method for decades.
If you're wondering what I said about Hitler. If he didn't get stuck in the Humanities and had today's ev psych, maybe he'd made some better choices about painting too many castles, unnecessary solutions and human experimentation. No one seems to talk about the Nazis and the English clinging to sociology, the English slow-walking Darwinian development. The English abuse of providence, religious story and metaphor and its relationship to disfunctionally behaviorally shapibg the Irish Catholics.
So yeah. Milgram. Good stuff. A personal bracelet to snap is a popular trick ammongst some musicians. I don't have much of a preparation problem, I've had a distracted by confusion about mechanics and provincial pseudo-science problem.
TLDL, no worries 😉
Great points throughout, and I think I agree with your argument.
The word frequency effect is a nice example—psycholinguistics does have a quite robust set of replicable empirical findings (to say nothing of psychophysics): it’s true that deleting a given paper shouldn’t necessarily cause us to change our minds much, and that is more a sign of the robustness of the effect across papers than a theoretical weakness.
I get the sense that Mastroianni’s frustration is more with the underlying theoretical scaffolding of the field (which I think connects to the claim that there are no “big ideas”), but I agree with you that the evidence marshaled in his original piece doesn’t entail theoretical inadequacies per se.
In my view a fundamental challenge is that psychology deals with slippery constructs that are hard to operationalize, and I don’t know whether that challenge is going away anytime soon.
In the moment of reading, I had EXACTLY the same reaction as Jason S. #3, especially, would be absolutely world-changing if we could only bring ourselves to TRULY believe it - as in actually internalizing its insight and acting on that instead of our more naive assumption of its opposite. For one thing, the ways in which so much of the practice of law is conducted would HAVE to be changed. Especially criminal law and the conduct of trials. #2 and #4 both "color" #3, when one thinks about what would be needed in the conduct of society writ large, were we actually to try to make the world better by acting on a belief in these results. So, I'm thinking that social psychology IS implicated, and quite heavily, in this list. Thinking harder about the list after the fact, it seems #7, too, COULD impact how we do regulatory law (but, so far, mostly doesn't). Finally, #5, and to a lesser extent, #6 and #9, could have much greater impact on social activity were we not so predisposed to ignore them as being in violation of popular contemporary political ideologies. So it's not really that these are not "world-changing" - it's that we won't LET them be.
I ended up writing about the replication crisis in psych this week after being (partially) inspired by this post! https://darbysaxbe.substack.com/p/cogsplained-by-my-ai-doppelganger
In my youth, I thought of being a psychologist. I had 56 course hours in psychology and needed 4 more in the field (+ a few required courses in other fields) to qualify for my undergraduate degree in psych and I was feeling quite like Adam Mastroianni. It seemed like every issue of Psych. Today has a new wunderkind psych theory to erase all mankind's psychological problems/
Knots was poetry and Being & Nothingness was philosophy, but they seemed to describe man's psychological better than any...basically a communication deficit that I didn't see any others addressing.
But Sarte went nuts (more or less)trying to justify his new found Marxism in the 30's, and the same year Knots appeared Kingsley Hall was shut down.
These guys (I thought) understand our prime mental turmoil better than any of the "cures" being offered but then when they turned to their own cures they had no more to offer. Well I dropped out of psychology to pursue philosophy.
But this doesn't seem to be Adam's problem quite as much. He seems to be less troubled by the theories of psychology and more about its efficacy and whether it matters if psychologists crib.
I agree it should matter, but I disagree that it doesn't. I think psychology has entered into becoming more scientific and failed, unsustainable ideas are becoming recognized when they can't be verified successfully and I think (perhaps wrongly) that only in the last twenty years have we had any way to verify psychological perspectives. So its in its infancy, only now, as a science, in my estimation and there will be more and more theories, papers, tests, procedures, and ideas of how the brain functions that will be able to be rejected if they cant be sustainably verified.
In short, what may be rejected isn't necessarily bad and that what can't be verified is being exposed more frequently is very beneficial towards psychiatry becoming a legitimate science.
P.S. Sartre is not generally recognized today as a psychiatrist, but he was a prominent psychology student in the 20's and early 30's who studied under most of the prominent post-Freudians in France, Berlin & Vienna. He also worked in the asylum in Paris for a few years. Being & Nothingness emerged out of his thesis to earn his Ph.D. in psychiatry, and it success led him to abandon completing the thesis/
I think Mastroianni is also making one other point that you didn't address (perhaps because it's sort of orthogonal to the other points about psychology more generally), which is this:
- Stapel was considered highly successful in his field (many papers, 10,000+ citations etc.)
- Stapel turns out not to be very important in his field (deleting his many papers and orphaning 10,000+ citations doesn't seem to overturn anything particularly significant)
- So are we defining "highly successful" completely wrongly?
Or as Mastroianni put it, "Every marker of success, the things that are supposed to tell you that you're on the right track, that you're making a real contribution to science—they might mean nothing at all."
And I think that this is a very significant question to ask. This is also a theme that Mastroianni has raised elsewhere, as have many others: is the pursuit of publication counts and citation numbers undermining the pursuit of good science? And this applies not only to individual researchers who want to publish and be cited, but also to publications who also measure themselves, at least in part, on how heavily they are cited.
I would suggest that psychology - and by the way, much of the rest of science - has succumbed to Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure". As academia has increasingly defined who is an important scientist by publication and citation numbers, those numbers became a goal in their own right, and the numbers became decreasingly useful as a measure of importance.
I don't mean to suggest an unusality that only indicates psychology, but from its earliest post-Freudian generation both behavioral studies and case histories were often questioned on whether they were fraudulent.
And yet, perhaps, Mastroianni may have a point to be said in his favor and markers of what is successful should be reevaluated continuously, not just after scandals like Watson's or Burt's fabricated twin studies in the 60's. So both bad methodology like Watson's and fabricated data such as Cyril Burt presented (both highly respected in their time) still persist to influence.
Therin is the real problem and what leads to both, psychiatrists sometimes want their results to be different than it is and they will alter or fudge data to make it so.
Not unique to psychiatry and much more of a political issue. What disturbs me more than anything, perhaps because it was one of the leading reasons I dropped out of psychology, was Burt's studies because they continue to be propagated by supremacists and there is no evidence that any follow-up" studies have been better.
Perhaps the greatest crisis in science as a whole, but certainly in those outside of chemistry and physics, is the expectation of certainty. The proliferation of statistical tools and professional publishing has led to an expectation that we can divine simple truths from complex systems like sociology, economics, human biology, nutrition, etc.
The issue is not that we have a replicability crisis, it is that we as a society have an epistemic crisis - we expect more "truth" than the world, our senses, and our instruments can provide.
Psychology is not dead because we cannot treat it like a Hubermann episode, and philosophy is not dead because we can't prove it through double-blind experiments. Complex systems rarely yield to simple truths, so our modes of contemplation have to calibrate to a realistic level of resolution.
I really like this exercise of trying to identify the most significant findings, and I'd agree that 1-6 are worthy candidates. 7-10 are interesting/useful but not field-changing.
BUT among the first six, all of those findings emerged prior to the 1980's, definitely not "in the last few decades". I'm sure you know that #4 "perceptual inference" goes back to Helmholz, before psychology was even called psychology.
I don't think change blindness is the most substantial illustration of the limitations of consciousness, but even that finding goes back to the 1970's, before ubiquitous peer review and other homogenizing practices.
In "recent decades" we've given more confidence to these six findings but mostly we've just been filling in the details---for example: understanding average developmental differences between 10 months and 14 months, but not overturning the fundamental theories/concepts that organized the paradigm in the first place.
So it feels like your examples actually underscore Adam's point.
Your list of 10 is really impressive (I noted this at the time, but I think it bears repeating - those are great insights).
One more thing. This may seem like a non-sequitor. We all know the Khmete Rouge horrifically smashed babies against trees, the woman taking off their glasses.
Was it the first born males? I ask that with no tinge or suggested relevence of anti-Semitism. I know all that other stuff. I also think Frankfurt Sociology owes many Jewish families some degree of an apology. They set-up the bowling pins for Hitler. I think that's maybe what Steve Pinker tries to manage in the community he at least in part writes for. You're both great.
Your article does not only serve as a response to Mastroianni's suggestions, but also an indicator to how psychology-related experiments should be scrutinized and appraised by researchers in general. If you originally have trust in a certain psychological principle, even if you find a single case of fraud, you would likely take into account countless other studies and researches that still back up the principle. Therefore, unless it's a case where the most representative research is attacked, or where the total sum of researches are unconvincing or shallow in the first place, Mastroianni's concerns should be inconsequential.
I think this shows how researchers can also fall vulnerable to confirmation bias. If you find a principle valid, the faulty case would simply be one outlier among countless other studies that say otherwise. However, if you have doubts in a principle in the first place, that would be when you would solely spotlight the single faulty research against the rest.
5,6,8 and 9 are findings from behavioral genetics that most psychologists consider debunked pseudoscience. Even w(o)ikipedia says the damns the whole field and labels it controversial.
This rocks!
Very interesting. Your top-10 findings list was especially interesting. I have always wanted to ask a psychologist if anyone in your field has made an attempt to falsify the "mimetic desire" theory discussed by René Girard. I recently read about his theory in his analysis of Shakespeare called Theatre of Envy, which I heard about in a Tyler Cowen podcast and I have no idea what to make of it because I'm not smart enough. But if it were true it would seem to be a top-10 type discovery and apparently Shakespeare discovered it. And if it were false then that would be a pretty big deal too since Girard has recently been gaining steam.