Readings
These are starter readings for Topic 9. Please also read the associated blog posts and listen to the podcasts.
Redefine Statistical Significance
Podcasts and links
Hertz Podcast 52 - Give p’s a chance (with Daniel Lakens)
Essay Topic - Should we redefine statistical significance?
In 2018, a group of scientists (Benjamin et al., 2018) published a paper in Nature Human Behaviour suggesting that the critical p-value for defining statistical significance should be changed from .05 to .005. This would, they claimed, reduce the rate of false-positive results in Psychology and go some way towards solving the Reproducibility Crisis. However, other scientists have argued that this solution is too simplistic and that what is really needed is a better understanding of what alpha levels mean (e.g. Lakens et al., 2018, de Ruiter et al, 2018). Still others have argued that we should abandon statistical significance altogether (e.g. McShane et al., 2019). What do you think? You are encouraged to make use of less conventional references such as blog posts in this essay. Starter readings can be found in Topic 9.
Some questions to consider:
- Do scientists really understand alpha values? Should they need to?
- What would be the consequences of abandoning null hypothesis significance testing (NHST) altogether?
- Would redefining significance at alpha = .005 disadvantage those from small or non-funded labs?
- Could this be addressed by measures to reduce publication bias, and/or large collaborative studies?
- If you were carrying out your Honours study (or if you are right now), what would you choose as your significance level?
Forum
For years, we’ve been somewhat mindlessly using the arbitrary cutoff of p < .05. Why do we use this level? Should we change it?
Benjamin and colleagues argue that we should change the default p-value to .005. Lakens argues that we should be more flexible with our alpha-levels but justify our choices. What do you think? Have you seen other arguments out there?
Post your thoughts below!