What is Open Science?

“Open science” is an umbrella term used to refer to the concepts of openness, transparency, rigor, reproducibility, replicability, and accumulation of knowledge, which are considered fundamental features of science”

— Crüwell et al. (2019), p.3

• a movement developed to respond to crisis in scientific research
  • lack of accessibility, transparency, reproducibility, and replicability of previous research
• transparency is key to all facets of Open Science
  • it allows for full evaluation of all stages of science
• Open Access, software, data, code, materials…

Systemic problem in science

• the combination of
  • publication bias
    • journals favour novel, significant findings
  • publish or perish
    • researchers’ careers depend on publications
• can/does/did lead to:
  • HARKing
    • Hypothesising After Results are Known
  • p-hacking
    • (re-)running analyses until a significant effect is found
  • replication crisis
    • pervasive failure to replicate previous research

Eight Steps to Open Science

Image source: Kathawalla et al. (2021) (all rights reserved)

Journal Club

• level: Easy

• e.g., ReproducibiliTea Berlin

  • discuss topics and share knowledge on Open Science Practices

Project Workflow

• level: Easy

• folder structure

  • how to sensibly set up your folders

• contained environments

  • using RProjects and the here package

• data management

  • establishing some data storage convention

• version control

  • e.g., git, GitHub/GitLab, OSF

Preprints

• level: Easy

• manuscript version publicly available

  • prior to peer review
  • during peer review
  • after publication

• allows for a wider audience

  • earlier feedback
  • actually increases citation count

• typically found on (psy)arXiv, OSF

Reproducible Code

• level: Medium

• with open source software (R, RStudio, packages)

• literate programming

• dynamic reports with Quarto/Rmarkdown

• reproducibility goes hand-in-hand with project workflow and data management

• ideally:

  • avoid GUI (Graphic User Interface with point-and-click, e.g., SPSS)
  • avoid propreitary software (paid licences, e.g., SPSS, Matlab)
  • use open software (e.g., R, Python)
  • use a programming language and include useful comments

Data sharing

• level: Medium

• publicly sharing your data

  • including raw data (if possible)

• allows for reproduction of analyses

• takes forethought and experience

• documentation and naming conventions are important

  • e.g., data dictionaries/codebooks

Transparent writing

• level: Medium

• transparency regarding

  • methods/procedure
  • hypotheses (confirmatory vs. exploratory)
  • data analyses

• an experiment plan or lab notebook are key!

Preregistration

• level: Medium

• a timestamped and (often) public plan of:

  • research questions
  • hypotheses
  • method
  • analyses

• clearly state intentions and predictions for confirmatory analyses

  • everything else is exploratory

• templates available on AsPredicted and the OSF

Registered Report

• level: Difficult

• submitting the introduction, methods, analysis plan to a journal before data collection

  • if accepted: publication regardless of the result

• a more detailed pre-registration, often with fully written sections

• much more time consuming before data collection can begin

  • journal acceptance can take months