MOOC Students Who Got Offline Help Scored Higher, Study Finds

One of the first things researchers have learned about student success in massive open online courses is that in-person, one-on-one teaching still matters.

For online learners who took the first session of “Circuits & Electronics,” the Massachusetts Institute of Technology’s hallmark MOOC, those who worked on course material offline with a classmate or “someone who teaches or has expertise” in the subject did better than those who did not, according to a new paper by researchers at MIT and Harvard University.

The research, published this week by the journal Research & Practice in Assessment, is one of the first peer-reviewed academic studies based on data from a MOOC. Advocates for the massive online courses have cited their potential value as engines of educational research.

The journal’s summer issue takes stock of MOOCs as a research medium and outlines an agenda for further study. The issue includes literature reviews of assessment methods, such as automated grading software, similar to those that many MOOC professors use in their massive courses.

In terms of hard data analysis, the published findings are fairly narrow. But there are some intriguing insights hidden in the 230 million interactions inventoried by the MIT and Harvard researchers, who in their study of “Circuits & Electronics” combined “clickstream” data with traditional data points like test scores and survey responses.

The research team, led by Lori Breslow, director of MIT’s Teaching and Learning Laboratory, tried to drill down to what types of demographic and behavioral factors contribute to student success in a massive online environment.

“On average, with all other predictors being equal, a student who worked offline with someone else in the class or someone who had expertise in the subject would have a predicted score almost three points higher than someone working by him or herself,” write the authors.

The correlation, described by the authors as the “strongest” in the data set, was limited to a single instance of a particular MOOC, and is not exactly damning to the format. But it nonetheless may give ammunition to critics who say human tutelage remains essential to a good education.

Other findings could also raise eyebrows. For example, the course’s discussion forum was largely the dominion of a relatively small group of engaged users; most students simply lurked. “It should be stressed that over 90 percent of the activity on the discussion forum resulted from students who simply viewed pre-existing discussion threads, without posting questions, answers, or comments,” the authors write.

At the same time, students referred to the discussion forum frequently when completing homework assignments or tackling examinations. As a resource for students doing graded work, the forum was more popular than the lecture videos, the tutorials, or the recommended textbook.

The authors of the MIT paper say that they hope to use these early insights, combined with other data sets, to understand which students are most likely to succeed in MOOCs, and why.