Why social science is difficult – and in favour of a slow(er) psychology…
Penicillin; developments in nuclear fusion; antiretroviral therapy: Science has given us much, and many of the breakthroughs can seem like magical solutions to complex and troublesome problems. Isn’t it ironic then, that science—social science included—has failed to come up with a silver bullet for its own systemic problems? After all, the much publicised replication crisis in psychology was in the early- to mid-2010s, about a decade ago now; why have we not found a once-and-for-all solution to resolve all the problems psychology—and indeed, the social sciences—have?
The replication crisis
If we’re being entirely honest, the term replication crisis may have been chosen as much for its shock journalism factor as for the fact that there was a genuine crisis of confidence in psychology. Certainly, I’m not the first person to assert that the replication crisis may be misunderstood. Arguments for why it is that science—more specifically, psychological science—may not replicate at the rate we expect range from the publish or perish culture in academia, to questionable research practices or fraud, to some authors even asserting that failure to replicate represents good science. In light of all of this, it’s no surprise that general concern about the “replication crisis” has risen substantially, if we take data from Google’s search trends – which indicates the peak search popularity as having occurred in September 2023 over the past decade. Clearly, the topic is more relevant today than ever.
Perhaps the question “why does this happen?” is best answered by “well, it’s difficult”.
Let’s pause for a minute, and take a deep breath.
If you open an introductory psychology textbook today, there’s a high likelihood that you will find at least some reference to the replication crisis. However, at the risk of coming across as dismissive or flippant, perhaps the question “why does this happen?” is best answered by “well, it’s difficult”. The whole process of enquiry in science, from the formulation of an idea to the generation of hypotheses, a formal research question, formulating a research paradigm, to actually collecting, synthesising and analysing the data is fraught with possible missteps. Any breakdown in this chain can lead to results that may seem promising, but in reality could be misleading. Science is not the product of magical moments of understanding and resolution, but the result of a slow, methodical process of formulating hypotheses and developing theories. Psychology, as a social science, brings an added level of challenge, even when we compare it to the natural sciences. Human behaviour is many times more complex than particles, and on top of that your subjects are active, cognitive beings whose cognition and behaviour are centralin the research you’re trying to do. Yes, physics has the observer effect too, but the social sciences are unique in that our theories, constructs, and research engage conscious beings who have their own experiences and cognitions. You can plan the perfect experiment, control for all possible confounds ideally, and still achieve very different results with individuals due to circumstances outside of your control. We cannot control the conscious experiences of the subjects of our studies outside or inside our laboratories, nor how these factors interact to produce complex outcomes. A finding which later proves difficult to replicate may not be false; it may simply be that the circumstances which combined to result in it did not reoccur. One of Milgram’s classic findings, in which up to 92% of passerby look up when observing confederates who were looking up at skyscrapers, displays a markedly different effect when replicated. The rate of gaze following saturates below 50%, and the effect of stimulus size displays a different effect. This does not mean that the original authors were wrong, or that they misrepresented their findings, simply that the original contextual features of the experiment were not – and cannot – be replicated.
The complexities of scientific research
Try as we might, there’s a sound argument to be had that we cannot come up with experimental stimuli that will be perceived as the same by everyone. This is by no means a defence of the bad methodology or dubious publishing ethics that may also play a part, but contexts and environments change, and people change with them.
Beyond that, there are structural, epistemic issues that are holding psychology as a discipline back. Publish or perish is a phenomenon that is well documented within academia, which can lead to issues such as p-hacking or even research fraud, in turn contributing to a biased literature. Journals that wish to retain high impact factors and publish exciting or unexpected findings are also a significant contributor to the aforementioned problem. Frequently, the very structures that uphold academic research seem to push for quantity over quality. Science is made even more difficult by the structural constraints. Funding is scarce, and research that promises attention-grabbing headlines is more likely to receive funding. Without sufficient funding, how do researchers create novel and rigorous methodologies? How do researchers collect high-power, representative samples?
There are structural, epistemic issues that are holding psychology as a discipline back
The way forward
So where does this leave us?
Well, maybe things are not as bad as they seem. There is evidence that psychology may be learning from its mistakes. The very fact that the replication crisis is being taught in introductory psychology classrooms, and that a concerted effort is being made by the scientific community to improve publishing ethics is evidence to that end. More importantly, it seems that we need to qualify some of the issues that have dogged psychological science. Actually, psychology’s replication rate is not that surprising, given that we are a social science. In fact, when we look at replication efforts, the standout article from the Open Science Collaboration in 2015 differs systematically from most other replication attempts in any of the social sciences. For instance, they looked at over 100 papers, and the replication rate they found of 36.1% is significantly lower than some other attempts to replicate studies within psychology.. In an essay that I wrote for a course, I compared replication rates across several fields in the social sciences here, and tried to contextualise the findings. It may not be perfect, but science by nature is slow and difficult.
Our field still has a long way to go when it comes to research methodology and practices. But these problems are multifaceted and complex, and there’s no quick solution for it. The alternative is to abandon the pursuit of understanding human cognition and behaviour in a holistic, well-rounded manner. There might not be a quick fix for psychology, but perhaps that’s okay.
Image credit: photo by Jay Galvin. Licensed with CC BY 2.0 DEED.
Leo is a third-year student of psychology at the RUG. He’s interested in social psychology, neuroscience, cross-cultural psychology, and the intersection of these topics. In his free time, he can be found cooking, shooting film, playing music, or reading philosophy.
He is currently studying counterfactual thinking, processes of political polarisation, and awe experiences.
