[LMU-OSC News] ReproducibiliTea 10.06 at 14:30: 'A Compendium and Simulation of p-Hacking Strategies'

[LMU Open Science Center News]

Hello everyone

The ReproducibiliTea team would like to invite you to the fifth meeting of our Journal Club. We are very happy to welcome Angelika Stefan (University of Amsterdam) as our guest. Angelika belongs to the Department of Psychological Methods within the Faculty of Social and Behavioural Sciences of the University of Amsterdam.

When? Friday, 10th of June 2022, 2:30pm-4:00pm
Where? Via Zoom: Meeting ID: 917 8852 1090 / Password: Replicate (https://lmu-munich.zoom.us/j/91788521090?pwd=QmJPeEZLcGtYeGlmeGFnazVoUVFzQT09)
Paper: Stefan, A., & Schönbrodt, F. D. (2022, March 16). Big Little Lies: A Compendium and Simulation of p-Hacking Strategies. https://doi.org/10.31234/osf.io/xy2dk
Procedure: Angelika Stefan, one of the paper’s authors, will start with a short talk about the paper (roughly 20-30 min). We will then continue with an open discussion in which you can contribute your own thoughts and questions.

It would be helpful to have read the announced paper, but there is no obligation to do so. As the paper for this session is quite long, we would suggest to focus on the introduction (pages 2-10), the intermediate summary (page 31) the evaluation of potential solutions (pages 37-45) and the discussion and conclusions (page 46-48). Additionally, we would recommend to have a look at one example from the twelve simulations (pages 10-31). We prepared the following key questions to guide you through the discussion and also provide some guidance while reading the paper. For each question, you can think about the suggestions from the paper, but also bring in your own ideas:

1. What do you think: How prevalent are p-hacking-strategies in your field/community (e.g. in student thesis, in published research)?
2. Which potential solutions (described on pages 37-45) do you find the most convincing?
3. Which questions about QRPs do you still have? Which questions do you think are not answered by the scientific community, yet (both empirical questions e.g. specific prevalence rates and theoretical questions)?

Below, you can find a short abstract for the introductory presentation by our guest: Angelika Stefan (University of Amsterdam)

We are looking forward to seeing you soon! Feel free to contact us if you have any questions and/or remarks.

Your organisation team
Laura Goetz, Stephan Nuding, Leonhard Schramm


--

Big Little Lies: p-hacking and effective countermeasures
Angelika Stefan

A decade ago, John et al. (2012) found a high prevalence of questionable research practices (QRPs) among psychological researchers. This result was confirmed by many other surveys among researchers in the social sciences. QRPs range from accidental mistakes to clear scientific misconduct and its prevalence undermined the credibility of the social sciences. One of the most prominent QRPs is p-hacking which includes a broad range of different strategies to render non-significant testing results significant. But if QRPs and especially p-hacking are widespread, as previous research suggests: How problematic are the different strategies in fact? How big is the impact of different p-hacking-strategies in detail? And what countermeasures are most effective?

In this talk, Angelika Stefan will give a short, general introduction to QRPs. She will discuss p-hacking in more detail and distinguish it from other QRPs. Then, she will show some examples of simulations from this session’s paper that examined the impact of different p-hacking strategies. Based on the simulations’ results, possible countermeasures and solutions will be introduced and discussed.

About the presenter:

Angelika Stefan is a researcher and doctoral candidate at the University of Amsterdam and belongs to the Department of Psychological Methods within the Faculty of Social and Behavioural Sciences. She finished her Masters in Economic, Organizational and Social Psychology in 2018 at LMU Munich. Her work focuses on the application of Bayesian Statistics and the general dealing with methodological flexibility in the Social Sciences. She is an expert for Open Science, led many workshops on Open Science-related topics and organizes the ReproducibiliTea at her university in Amsterdam.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrz.de/pipermail/lmu-osc/attachments/20220603/cc3c0bd9/attachment.htm>