Megastudy

A megastudy or mega-study is a research study in which a large number of different treatments or interventions are tested at the same time, on the same sample or similar samples, using a common outcome measure, and using the same experimental protocol.

This research design ensures the outcomes across interventions are comparable. Additionally, due to generous inclusion of various interventions in the study, megastudies may be less prone to publication bias, where the interventions expected to be effective are more likely to be studied and the interventions found to be ineffective are underreported due to the file-drawer problem.

Megastudy examples

• Exercise encouragement
• Vaccination nudges
• Strengthening of democratic attitudes
• Facial analysis
• Interventions against climate change

Many-lab studies

The megastudy technique can be combined with the many-labs approach, so that teams of researchers from across the planet conduct the same experiment locally.

Megastudy criticisms

• Statistical power: While the overall megastudy sample size may be large, the sample size per intervention may be relatively small, leading to underpowered designs with wide confidence intervals. As a result, while interventions may be comparable, their relative ranking by outcome measure may be noisy. Increased sample size can help address this issue.

• Lack of theory: On the one hand, the megastudy technique may be considered a form of "fishing expedition" for what interventions have strongest effect on the outcome measure, without much theory building. On the other hand, such theory-free approach can help uncover unexpected findings, mitigate publication bias, and form new theories.