multiverse analyses: lecture + workshop, March 14 and 15
Dear quantitative researchers We are pleased to announce that a lecture and a workshop on multiverse analyses are open for registration! Participants to the (solely in person) workshop need to attend the lecture prior, but that (hybrid) lecture is open to all! Hybrid lecture: "Increasing transparency through preregistration and/or multiverse analyses" by Prof. Dr. Andrea Hildebrandt (Carl von Ossietzky Universität Oldenburg) Thursday March 14, 16:00-17:00, Munich, Leopoldstraße 13, room 2201<https://www.lmu.de/raumfinder/#/building/bw0601/map?room=060202201_> and on Zoom Registration: https://www.pretix.osc.lmu.de/lmu-osc/multiverse-L/ Abstract: Pre-registering an analysis plan means committing to analytical steps without knowing the results of the research. Multiverse analysis means committing to multiple, defensible analytical steps and investigating the robustness of the results across analytical choices. Assuming that all listeners are familiar with 'The preregistration revolution' (Nosek et al., 2018, PNAS), after a brief introduction to the rationale for multiverse analysis, I will argue in this talk that the two proposed tools for the credibility revolution (pre-registration and multiverse analysis) should be used together. In person workshop: "Making the black box transparent: Multiverse analysis and visualizations in R" by Prof. Dr. Andrea Hildebrandt and Dr. Cassie Short (Carl von Ossietzky Universität Oldenburg) Friday March 15, 9:00-12:00, Munich, Leopoldstraße 13, room 2201<https://www.lmu.de/raumfinder/#/building/bw0601/map?room=060202201_> Registration: https://www.pretix.osc.lmu.de/lmu-osc/multiverse-W/ Abstract: To describe the multitude of researchers' methodological choices when analyzing data, Gelman and Loken (2013) coined the term "garden of forking paths". Multiple defensible alternative options are available for selection throughout the study design, data preprocessing and data analysis workflow. In line with a traditional analysis approach, researchers will select one sole workflow of defensible options and often do not specifically disclose how the decisions were made. However, a multiple comparison problem occurs even if only one constructed dataset following one single workflow of decisions is used for statistical inferences. The reason is that theoretically a large variety of workflows are possible, but researchers do not correct their hypothesis tests for potential comparisons that they did not explicitly carry out. Recently, statistical approaches have been proposed to explore and integrate multiple inferential results accumulated along forking paths. Multiverse analysis (Steegen et al., 2016) refers to repeated hypothesis testing on the multitude of datasets resulting from different defensible decisions regarding, for example, variable selection, categorization, transformation, outlier selection, etc. After combining all potential choices by simple rules of combinatorics, contradictory combinations are eliminated. Datasets created on the basis of the remaining set of combinations are then submitted to statistical analyses along a looped pipeline. Results provide as many outcomes of statistical tests as datasets created in the multiverse. These can then be visualized to understand how statistical conclusions depend on methodological decisions or they can be statistically integrated. In this workshop, unexperienced attendees who attended the lecture on the previous day will be guided through an implementation example of multiverse analyses on the R Software for Statistical Computing. There will be time for exercises and discussions. Workshop materials and necessary R packages will be shared in advance and participants are expected to join the workshop with their own laptops. Target audience: Scientists from any discipline who apply statistical models in the framework of regression analysis, without prior experience with multiverse analysis implementations. Prerequisites: Basic statistics (including GLM) and statistical programming skills in R are required. Best wishes Malika -- Dr. Malika Ihle (she/her) (pronounciation<https://namedrop.io/malikaihle>) LMU Open Science Center Coordinator Website: https://www.osc.lmu.de<https://www.osc.lmu.de/> Mailing list: https://lists.lrz.de/mailman/listinfo/lmu-osc Bluesky: https://bsky.app/profile/lmu-osc.bsky.social LinkedIn: https://www.linkedin.com/company/lmu-open-science-center Mastodon: https://scicomm.xyz/@lmu_osc Twitter: https://twitter.com/lmu_osc Please note that I will only work in the afternoons from 08.01. to 28.02.2024.
Dear all This is a reminder that this lecture and workshop are taking place this week - you can still register! Hybrid lecture: "Increasing transparency through preregistration and/or multiverse analyses" by Prof. Dr. Andrea Hildebrandt (Carl von Ossietzky Universität Oldenburg) Thursday March 14, 16:00-17:00, Munich, Leopoldstraße 13, room 2201<https://www.lmu.de/raumfinder/#/building/bw0601/map?room=060202201_> and on Zoom Registration: https://www.pretix.osc.lmu.de/lmu-osc/multiverse-L/ In person workshop: "Making the black box transparent: Multiverse analysis and visualizations in R" by Prof. Dr. Andrea Hildebrandt and Dr. Cassie Short (Carl von Ossietzky Universität Oldenburg) Friday March 15, 9:00-12:00, Munich, Leopoldstraße 13, room 2201<https://www.lmu.de/raumfinder/#/building/bw0601/map?room=060202201_> Registration: https://www.pretix.osc.lmu.de/lmu-osc/multiverse-W/ Best wishes Malika -- Dr. Malika Ihle (she/her) (pronounciation<https://namedrop.io/malikaihle>) LMU Open Science Center Coordinator https://www.osc.lmu.de<https://www.osc.lmu.de/> From: lmu-osc <lmu-osc-bounces@lists.lrz.de> On Behalf Of LMU Open Science Center News Sent: Tuesday, 27 February 2024 08:28 To: lmu-osc@lists.lrz.de Subject: [LMU-OSC News] multiverse analyses: lecture + workshop, March 14 and 15 Dear quantitative researchers We are pleased to announce that a lecture and a workshop on multiverse analyses are open for registration! Participants to the (solely in person) workshop need to attend the lecture prior, but that (hybrid) lecture is open to all! Hybrid lecture: "Increasing transparency through preregistration and/or multiverse analyses" by Prof. Dr. Andrea Hildebrandt (Carl von Ossietzky Universität Oldenburg) Thursday March 14, 16:00-17:00, Munich, Leopoldstraße 13, room 2201<https://www.lmu.de/raumfinder/#/building/bw0601/map?room=060202201_> and on Zoom Registration: https://www.pretix.osc.lmu.de/lmu-osc/multiverse-L/ Abstract: Pre-registering an analysis plan means committing to analytical steps without knowing the results of the research. Multiverse analysis means committing to multiple, defensible analytical steps and investigating the robustness of the results across analytical choices. Assuming that all listeners are familiar with 'The preregistration revolution' (Nosek et al., 2018, PNAS), after a brief introduction to the rationale for multiverse analysis, I will argue in this talk that the two proposed tools for the credibility revolution (pre-registration and multiverse analysis) should be used together. In person workshop: "Making the black box transparent: Multiverse analysis and visualizations in R" by Prof. Dr. Andrea Hildebrandt and Dr. Cassie Short (Carl von Ossietzky Universität Oldenburg) Friday March 15, 9:00-12:00, Munich, Leopoldstraße 13, room 2201<https://www.lmu.de/raumfinder/#/building/bw0601/map?room=060202201_> Registration: https://www.pretix.osc.lmu.de/lmu-osc/multiverse-W/ Abstract: To describe the multitude of researchers' methodological choices when analyzing data, Gelman and Loken (2013) coined the term "garden of forking paths". Multiple defensible alternative options are available for selection throughout the study design, data preprocessing and data analysis workflow. In line with a traditional analysis approach, researchers will select one sole workflow of defensible options and often do not specifically disclose how the decisions were made. However, a multiple comparison problem occurs even if only one constructed dataset following one single workflow of decisions is used for statistical inferences. The reason is that theoretically a large variety of workflows are possible, but researchers do not correct their hypothesis tests for potential comparisons that they did not explicitly carry out. Recently, statistical approaches have been proposed to explore and integrate multiple inferential results accumulated along forking paths. Multiverse analysis (Steegen et al., 2016) refers to repeated hypothesis testing on the multitude of datasets resulting from different defensible decisions regarding, for example, variable selection, categorization, transformation, outlier selection, etc. After combining all potential choices by simple rules of combinatorics, contradictory combinations are eliminated. Datasets created on the basis of the remaining set of combinations are then submitted to statistical analyses along a looped pipeline. Results provide as many outcomes of statistical tests as datasets created in the multiverse. These can then be visualized to understand how statistical conclusions depend on methodological decisions or they can be statistically integrated. In this workshop, unexperienced attendees who attended the lecture on the previous day will be guided through an implementation example of multiverse analyses on the R Software for Statistical Computing. There will be time for exercises and discussions. Workshop materials and necessary R packages will be shared in advance and participants are expected to join the workshop with their own laptops. Target audience: Scientists from any discipline who apply statistical models in the framework of regression analysis, without prior experience with multiverse analysis implementations. Prerequisites: Basic statistics (including GLM) and statistical programming skills in R are required. Best wishes Malika -- Dr. Malika Ihle (she/her) (pronounciation<https://namedrop.io/malikaihle>) LMU Open Science Center Coordinator Website: https://www.osc.lmu.de<https://www.osc.lmu.de/> Mailing list: https://lists.lrz.de/mailman/listinfo/lmu-osc Bluesky: https://bsky.app/profile/lmu-osc.bsky.social LinkedIn: https://www.linkedin.com/company/lmu-open-science-center Mastodon: https://scicomm.xyz/@lmu_osc Twitter: https://twitter.com/lmu_osc
participants (1)
-
LMU Open Science Center News