Evaluating Columbia University’s Frontiers of Science course

Frontiers of Science is a course offered as part of Columbia University’s Core Curriculum. The course is controversial, with some people praising its overview of several areas of science, and others feeling that a more traditional set of introductory science courses would do the job better.

Last month, the faculty in charge of the course wrote the following public letter:

The United States is in the midst of a debate over the value of a traditional college education. Why enroll in a place like Columbia College when you can obtain an undergraduate degree for $10,000 or learn everything from Massive Open Online Courses? In more parochial terms, what is the value added by approaches such as Columbia’s Core Curriculum? Recently students in our Core Course, Frontiers of Science (FoS), provided a partial answer. The FoS faculty designed a survey to gauge the scientific skills and knowledge of the Class of 2016 both before and after taking FoS. In an assembly held during orientation week last August, 966 first-year Columbia College students answered questions covering basic skills such as statistics, probability, and the reading and analysis of graphs, as well as content to be taught during the fall session of FoS. For the 519 College students taking FoS in the fall semester, the same survey was administered again as part of the final exam. The mean score for the initial orientation survey was less than 28%. The mean score for the same questions at the end of the semester was 76%! (The margin of error in both cases was ±1.0%.) To control for gains independent of FoS, 167 first-year College students, who did not take FoS in the fall, answered the same survey questions again at the start of the spring semester. The mean score of those students was 31%, not substantially greater than they and their classmates had scored on the survey during orientation. This type of research always has its limitations. However, the results with and without FoS are so different that the conclusion is inescapable. The scientific habits and knowledge that FoS imparts are new to, and effectively learned by, first-year students after one semester of intense study . . .

This sounds pretty good, especially the part about “basic skills such as statistics, probability, and the reading and analysis of graphs.” My first thought is that, if these skills are so important, maybe all the students at Columbia should be taking a course in probability and statistics! But then I remembered that our intro stat course isn’t so great (I know, I’ve taught it several times), so maybe it’s just as well if some biologists, physicists, etc., create a new statistics module from scratch. Seriously, I have lots of ideas of how we could teach intro prob/stat better, but when I actually try to do it, I get all tangled in the details. So I can’t very well object to outsiders taking a shot at it. As users of statistics, they might have a better idea than I do of how to teach the subject.

Similarly, I wouldn’t be surprised if the engineers could teach a better intro physics class than the physicists could. And it could well make sense to have biologists teach first-year chemistry, and psychologists teach first-year biology.

What, then, would the statistician teach? First year math, of course. It would make the mathematicians cringe, but it might be closer to the students’ own level. Statisticians are users of math, and we could teach the subject from a user’s perspective. Just as, arguably, the collection of scientists who run Frontiers of Science might be teaching probability and statistics in a way more useful to freshman than whatever my colleagues and I in the stat department could come up with.

OK, now that I’ve established my complete acceptance of the idea that this team can, and perhaps should, be teaching probability and statistics to Columbia’s freshmen, let me say that there are a bunch of things about the above-quoted letter that bother me.

First, the economic basis for their argument: “Why enroll in a place like Columbia College when you can obtain an undergraduate degree for $10,000 or learn everything from Massive Open Online Courses?”

It’s not new that you can obtain an undergraduate degree for $10,000! Until relatively recently, in-state tuition at state schools was cheap, pretty much everywhere. At Columbia you get the top professors teaching what they know best. But that argument doesn’t work for Frontiers of Science, which features non-statisticians teaching statistics, and a particle physicist who strips off his clothes to get students to recognize that nuclear weapons proliferation is scary.

As for online education, it’s great, but if you can “learn everything” from a mook, you can learn everything by buying a stack of textbooks. Or maybe not. I just don’t see that the results from this study have anything to do with mooks, one way or another.

Second, the pre-test and post-test. This actually does seem like a good idea, and I remain embarrassed that my statistics colleagues and I don’t do this in our own classes. That said, “The margin of error in both cases was ±1.0%.” Are they kidding? And “the conclusion is inescapable”?? We’d need to know more about the post-test conditions of those 167 freshmen who were not in the course. One student writes that the students in Frontiers of Science were given “premade cheat sheets during the final” with formulas. Maybe the 167 controls were given the cheat sheets too, but such sheets might not mean much without some prior coaching.

What was on the quiz? Nicholas Christie-Blick, one of the authors of the letter quoted above, writes, “The questions and answers will be released. . . . Nor does it matter much what the questions are. What matters is that they are representative of the range of skills in scientific thinking that we call Scientific Habits.”

I’d say something snarky here, but then again I remember that my own intro stat course hasn’t always gone so well, nor is it clear how much my students ever learned.

So, in quick summary, I’m suspicious of the claims of efficacy of the Frontiers of Science course. But I’m sympathetic with their efforts to measure anything at all. Measurement in education is tough.

The post Evaluating Columbia University’s Frontiers of Science course appeared first on Statistical Modeling, Causal Inference, and Social Science.