Online Assessment: What do difficulty, correlation, discrimination, etc. in the question analysis mean?

Difficulty (or p value) should probably be called easiness as a value of 1 indicates that all students got the question correct and a value of 0 indicated that no participant gave the correct answer. It is calculated by dividing the mean score on a question by the maximum possible. A good rule of thumb is that difficulty should be around the pass mark of your assessment.

Item Discrimination, where available, is calculated by taking the top and bottom 27% of students, based on overall score in the assessment, and subtracting the fraction of bottom group who gave the answer from the fraction of the top group who gave the answer, giving a range of -1 to 1. So:

• A positive item discrimination means a higher proportion of people in the top group chose the answer than in the bottom group. A high positive value for the correct answer generally means the question is a good discriminator, which is what we want (but is difficult to achieve!). A positive discrimination for an incorrect answer may indicate a problem, but could just mean that it is a good distractor.
• An item discrimination of 0 means the same number of people from each group gave the answer, so the answer doesn’t discriminate at all. Questions where everyone got the correct answer will always have a discrimination of 0.
• A negative item discrimination means a higher proportion of people in the bottom group chose the answer. This would be expected for an incorrect answer. A negative discrimination on a correct answer may indicate something is wrong, as the ‘good’ students are not choosing the correct answer.

Item-total correlation discrimination uses a Pearson product moment coefficient to give the correlation between the question score and the assessment score. Higher positive correlation values indicate that participants who obtain high question scores also obtain high assessment scores and that participants who obtain low question scores also obtain low assessment scores. This is what we want. Values below one indicate potentially worrying low scoring participants doing well on this question and/or high-scoring students doing badly. So, low values for questions here could indicate unhelpful questions that are worth looking at in more detail.