General: Is educational technology really worth it? Is there any evidence of its usefulness?

The situation IN MSD at OXFORD

In Oxford’s teaching environment, where face to face and small group teaching are the norm, the main roles for educational technology are probably:

• giving students control over their learning - where, when, how and what e.g access to self-test quizzes from home, access to MedLearn while waiting for a delayed consultant, access to those slides that they didn’t have time to look at in the practical class, access to recorded lectures to have another go at understanding that tricky bit in the middle.
• simulating situations not available in the real world - too difficult, costly or dangerous
• making learning fairer e.g removing marker variability through online assessment, alternative means of access to material
• facilitating collaboration/communication – audience response, group work, peer marking, etc.
• reducing administrative burden

GENERAL EDUCATIONAL TECHNOLOGY

However, there is a considerable body of research out there on the efficacy of educational technology.

In a meta-analysis of 25 meta-analyses with minimal overlap in primary literature, encompassing 1,055 primary studies, Tamin et at. (2011) found that educational technology led to a mean 12% higher in attainment vs controls but that:

“effect sizes pertaining to computer technology used as “support for cognition” were significantly greater than those related to computer use for “presentation of content.” Taken together with the current study, there is the suggestion that one of technology’s main strengths may lie in supporting students’ efforts to achieve rather than acting as a tool for delivering content.”

Higgins et al. (2012) carried out a similar meta-analysis of 48 meta-analyses. Although it was based on research in schools, its principles are equally applicable in HE. They were more cautious about the general impact of educational technology:

“the correlational and experimental evidence does not offer a convincing case for the general impact of digital technology on learning outcomes”

But did find evidence for the benefits of certain approaches to the use of educational technology:

• Collaborative use of technology (in pairs or small groups) is usually more effective than individual use, though some pupils, especially younger children, may need guidance in how to collaborate effectively and responsibly.
• Technology can be as powerful as a short but focused intervention to improve learning, particularly when there is regular and frequent use (about three times a week) over the course of about a term (5 - 10 weeks). Sustained use over a longer period is usually less effective at providing this kind of boost to attainment.
• Remedial and tutorial use of technology can be particularly practical for lower attaining pupils, those with special educational needs or those from disadvantaged backgrounds in providing intensive support to enable them to catch up with their peers.
• In researched interventions, technology is best used as a supplement to normal teaching rather than as a replacement for it. This suggests some caution in the way in which technology is adopted or embedded in schools.

Higgins, S., Xiao, Z., and Katsipataki, M., 2012. The Impact of Digital Technology on Learning: A Summary for the Education Endowment Foundation. University of Durham/Education Endowment Foundation. 52pp.

Tamim, R. M., Bernard, R. M., Borokhovsi, E., Abrami, P. C., & Schmid, R. F., 2011. What Forty Years of Research Says About the Impact of Technology on Learning: A Second-Order Meta-Analysis and Validation Study. Review of Educational Research, 81(1), 4–28.

To support these more general studies, several studies have looked at the impact of individual technologies.

Audience response

Kay, R.H and LeSage, A., 2009. Examining the benefits and challenges of using audience response systems: A review of the literature. Computers and Education, 53(3): 819-827

Preszler, R.W., Dawe, A., Shuster, C.B. and Schuster, M., 2007. Assessment of the effects of student response systems on student learning and attitutudes over a broad range of biology courses. CBE-Life Sciences Education, 6(1): 29-41

Peer ASSESSMENT

Although a review of secondary school studies, and also including self-assessment alongside peer assessment, in their systematic review Sebba et al., 2008, reported that most studies reported some positive outcomes including: pupil attainment (9 out of 15 studies); pupil self esteem (7 out of 9 studies) and increased engagement with learning (17 out of 20 studies).

A more pertinent, but much smaller, study of 71 medical students by Eldredge et al., 2013 found that students who had been trained in peer assessment had higher mean scores (45/7 out of 49) on a formative test than did students in the control group (43.5 out of 49) with a respectable p-value of 0.06.

Sebba, J., Deakin Crick, R., Yu, G., Lawson, H and Harlen, W., 2008.Systematic review of research evidence of the impact on students in secondary schools of self and peer assessment - Report. Social Science Research Unit, Institute of Education, University of London. 29pp.

Eldredge, J.D., Bear, D.G., Wayne, S.J. and Perea, P.P., 2013. Student peer assessment in evidence-based medicine (EBM) searching skills training: an experiment. J Med Libr Assoc., 101(4): 244–251