Cookies on this website
We use cookies to ensure that we give you the best experience on our website. If you click 'Continue' we'll assume that you are happy to receive all cookies and you won't see this message again. Click 'Find out more' for information on how to change your cookie settings.
Skip to main content

This classroom based MSD Skills training course consists of two hour sessions on two separate days. It is suitable for MSD researchers with basic statistical knowledge wanting to learn how to simulate experimental data and then apply standard statistical tests.

This course takes place over two separate days and participants are expected to attend both sessions.

29 October 2019 @ 15:00 - 17:00

31 October 2019 @ 15:00 - 17:00

PLEASE NOTE: you will need to install the free software in the links in the pre-course material at least a week in advance of the course. Once you are booked on the course, the pre-course material is available in 'My resources' in the booking system.

COURSE AIM

In recent years, there has been an increasing focus on the 'replication crisis', with evidence that much published research is not robust. There are many reasons for this: in this course, the focus will be on the importance of having a deep understanding of the basic statistical methods that are commonly used for null-hypothesis significance testing. Human cognition is not well-suited to thinking about probability, and so if researchers are simply trained to apply statistical tests, they may do so in a way that is very likely to generate non-replicable findings. A good way to gain a deep understanding of the nature and limitations of statistical methods is to simulate experimental data with known characteristics, and then apply standard statistical tests. This course will be in two parts, and attendees should attend both of these. 

In session 1, the focus will be on gaining a deep understanding of what a p-value means, and in particular how easy it is to obtain 'significant' results from null data if there is a flexible approach to data analysis.

No prior knowledge of coding is required, and the initial exercises will use Excel, to illustrate the basic principles that are involved in data simulation. All attendees should bring a laptop that runs Excel. Subsequent exercises will use the R programming language. No prior knowledge of R is required, and indeed this course can act as a gentle introduction to R. However, it is important that all attendees have R, R studio and some related packages and scripts already installed: the instructions for doing this are in the document 'Initial installations'. Attendees on this course will also need to have a basic knowledge of statistics, and be familiar with t-tests and correlations. One aim or the course is to provide attendees with a more intuitive understanding of what these statistics can and can't tell us.

In session 2, the focus will be on gaining a deep understanding of selecting an appropriate sample size to show effects of interest, i.e. statistical power. To do this, we will build on what was learned in part 1 of the course, to create simulated datasets which have variables with associated effects of known size. We will then take different-sized samples and consider how often our statistical analysis is able to reject a null hypothesis and detect that there is a true effect.

It is assumed that attendees will have attended part 1 of the course, and so will have R, R studio and some related packages and scripts already installed: the instructions for doing this are in the document 'MEDS - Pre-course material for Simulating data to improve your research.docx'. This is available in 'My Resources' in CoSY once you are booked on the course.

COURSE FORMAT

Two sessions of interactive lectures followed by hands-on exercises delivered by Dorothy Bishop, Experimental Psychology, Oxford.

COURSE CONTENT

At the end of session 1, attendees should be able to:

  • Simulate distributions of variables with known means and standard deviations, using R.
  • Understand why it is important to apply corrections for multiple statistical tests.
  • Understand the value of simulating data to check out an analysis plan prior to running an experiment

At the end of session 2, attendees should be able to:

  • Simulate distributions of variables with known means and standard deviations, using R.
  • Understand why doing research with an insufficient sample size is wasteful and can result in false acceptance of a null hypothesis
  • Understand how to use simulation to do a power analysis with a complex experimental design

COURSE LENGTH

Two 2 hour sessions

 

NUMBER OF PARTICIPANTS

20

 

REFERENCE

Bishop, D. V. M. (2019). World View: Rein in the four horsemen of irreproducibility. Nature, 568, 435. doi:10.1038/d41586-019-01307-2