Open Science

Inclusive Open Science Article False-Positive Psychology Article Measurement Issues Article

“If the point of scientific research is to discover truths about the world and enact change, competition is not the way forward. Collaboration is.”

Open science and scholarship is the way forward in research disciplines. It’s obvious that there is a major problem in the ways we conduct, evaluate, and disseminate research. Studies aren’t replicating, both in psychology and other fields . There is a whole host of potential explanations for this: our publishing systems, incentive structures, accidental misuse of statistical tools, and of course, malicious misrepresentation of data. But that doesn’t mean that there aren’t solutions out there, but they may be difficult to implement, and implementation can have unintended consequences for early career researchers or individuals who aren’t at wealthy and powerful institutions.

I support implementation of a few easy solutions that everyone can implement:

  • Pre-registration of eligible studies. While exploratory research is incredibly valuable, it’s incredibly important to justify decisions before they’re made, in order to avoid p-hacking or HARK-ing.
  • The 21-word solution. This simple sentence, added to the methods sections of emperical research, reads: “We report how we determined our sample size, all data exclusions (if any), all manipulations, and all measures in the study.”
  • Preprints. Posting versions of articles before they are peer reviewed on public repositories such as PsyArXiv increases transparency, visibility, and creates a time-stamped credited version of when a research project was conducted. Preprints also help mitigate some of the issues around publication bias.
  • Sharing of data, code, and materials. Along with preprints, sharing data and code (of course, when applicable) makes it easier for individuals to replicate studies, check the quality of code, and conduct meta-analytic studies.

I do also want to emphasize the importance of equity in open science. Open science practices are risky for many researchers to undertake, and they often are at odds with incentive structures and power dynamics. It’s important to realize that not all individuals are able to take all of the steps that they want to, and not to hold that against individual researchers. Of course, when given the opportunity, it is better to preregister, to post data, materials, code, and preprints, even to conduct registered reports . Putting researchers down for not following certain practices or dismissing studies for coming out of certain labs or certain countries without warrant is not ok. Open science should focus on lifting up all researchers as opposed to putting certain people down. After all, if the point of scientific research is to discover truths about the world and enact change, competition is not the way forward. Collaboration is.