Violent science teacher makes ridiculously unsupported research claims, gets treated by legislatures/courts/media as expert on the effects of homeschooling

Paul Alper shares this horrifying news story by Laura Meckler:

Brian Ray has spent the last three decades as one of the nation’s top evangelists for home schooling. As a researcher, he has published studies purporting to show that these students soar high above their peers in what he calls “institutional schools.” . . .

His influence is beyond doubt. He has testified before state legislators looking to roll back regulations. Judges cite his work in child custody cases where parents disagree about home schooling. His voice resounds frequently in the press, from niche Christian newsletters to NPR and the New York Times. As president of the National Home Education Research Institute, he is the go-to expert for home-school advocates looking to influence public opinion and public policy, presenting himself as a dispassionate academic seeking the truth.

OK, now for the bad news:

Taken as a whole, the academic literature shows mixed academic outcomes for home schooling: Some studies find benefits; others show deficiencies. Nonetheless, Ray’s work, which concludes home-schoolers score far above public school students on standardized tests, has been widely cited for many years. . . .

Hey, how did that happen?

Ray, 69, received his master’s degree in zoology and earned a PhD in science education at Oregon State University, thinking he might be a science teacher. But soon he grew interested in home schooling — both for his own children and professionally. Oregon State University rejected Ray’s proposal to study home education for his dissertation, but he began collecting data. In 1985, he started a journal called the Home School Researcher. Around this time he met Michael Farris, who had co-founded the HSLDA and was beginning to build the Christian home-schooling movement. “He said, ‘Brian, you’re already an expert. But the moment you get your PhD handed to you, call me, and we’ll bring you into court as an expert witness,’” Ray recalled . . .

Ray’s decades of research demonstrate one point beyond dispute: Some portion of home-schooled students do very well academically, and home education can be successful for some children. The question is whether those children are representative of home-schoolers and whether research supports his oft-stated contention that home-schoolers perform 15 to 30 percentile points better than public school students. (Recently, he said, he has revised his estimate to 15 to 25 points.)

Ray’s studies have included thousands of students across the country, often recruited with the help of HSLDA. He compares their results on standardized tests to those of public school students, consistently finding home-schoolers with higher scores in all subjects. He finds home-schoolers register as high as the 80th percentile — sometimes even higher — meaning their average score is better than at least 80 percent of traditionally educated test-takers. . . .

In an interview, he offered some possible explanations. Home education provides small classes, more feedback from adults and freedom from bullying, he said. If anything, Ray told The Post, standardized tests are written to test what students are taught in public school, so home-schoolers are at an inherent disadvantage.

Meckler continues:

Critics cite numerous problems with Ray’s approach: These tests are optional in the vast majority of the country, and many home-schooled students don’t take them. The ones who don’t might have scored far worse if they had been required to sit for exams, as public school students are. Many students take the exams at home, which might offer advantages over public school test-takers who face a controlled environment. And parents had to opt into Ray’s studies, potentially skewing his sample further. Demographic information collected as part of Ray’s research showed almost all students in his samples were White, Christian and came from two-parent married families. Their parents were more educated than average. In short, they were the type of students who tend to do well no matter where they are educated. . . .

What does he say about that?

In an interview, Ray responded that all studies have “limitations,” but he said that does not make his results invalid. He also said he has worked to include more representative samples and demographics in his research, saying methods “mature over time within a field.” . . . Asked whether it’s possible that students who do well in his studies would do well in any setting, given their demographic advantages, Ray replied, “That’s a reasonable hypothesis.”

But:

He dispenses with the caveats when talking about his results to legislators, courts, journalists and the public. In a 2005 book he wrote about home schooling aimed at general readers, Ray repeatedly cited his studies’ findings with none of the cautions included in academic papers. He mentions none on his website, either. He takes the same approach with the press. “The research said over and over again,” he told the Pensacola News Journal in 2012, “that these young people are performing above average and on average they’re surpassing public school students.” . . .

A 2009 “progress report” focused on his findings said, “Homeschoolers are still achieving well beyond their public school counterparts,” a generalization that does not take into account the demographics of his sample. . . .

Last year, New Hampshire lawmakers were considering whether to remove a requirement that home-schooled students show “reasonable academic proficiency,” which was defined as scoring in the 40th percentile or higher on state exams. Ray was among those making the case for reduced regulation. Ray testified that “40 years of research” by “various scholars” finds home-schooled students “typically outperform public school students by 15 to 30 percentile points.” . . .

The story gets much much worse (see below), but here I want to comment on something else, which is, on a research level, that all of this is just slightly worse than . . . some of the highest-profile academic research on education being conducted at top universities!

Remember that claim that early-childhood intervention increased later adult earnings by 42%? That was another case of selection bias (see also here), albeit a bit less blatant than what Ray did. It makes sense that, when a team of credentialed economists make an error, it’s a more subtle error than that made by someone trained in zoology and science education.

That said, there’s a disturbing consonance between the zoologist’s “If anything . . .” and the economist’s “Their relatively small sample sizes actually speak for — not against — the strength of their findings.” The common thread: an absolute eagerness to explain away problems, a focus not on probing and figuring our what could go wrong but rather an insecure defensiveness. Lots of glib answers (in the zoologist’s case, supported by irrelevant Bible quotations; in the economist’s case, supported by irrelevant mathematical arguments), not much thought. And, of course, policy impact. Lots of hype, lots of NPR appearances, the whole deal.

So, yeah, this world of bogus homeschooling statistics is a kind of funhouse-mirror version of prestige social science.

Just to be clear, I’m not saying that all education research is bad. Education research is important, and there’s some good stuff! The bad thing is this push-a-button, take-a-pill model, the idea that you can do an experiment on 130 kids and discover a 42% effect. This homeschooling stuff is worse, as they’re not even trying to do things right—but the consequences are kind of the same. They make big claims, they get media attention, respectful treatments from courts and legislatures who are just looking for someone credentialed to give them the message they want to hear.

In human terms, though, this is all much worse.

Also from the news article:

[Ray’s oldest daughter, Hallie Ray Ziebart] said her father taught her almost no math, routinely required her to work long hours for his nonprofit institute during school days, and whipped her and her siblings with switches and other objects when they disobeyed his orders. Her allegations were echoed by two of her siblings and by four others who spent time at their home. Some of her charges are bolstered by journals she kept at the time. . . . She said that she was told, for instance, that the slave trade was “meant for evil but God made it for good” and that things worked out for enslaved people “because they got to be Christians.”

Here are some horrible details:

Ziebart has spoken publicly about what she calls physical abuse — being hit with sticks, wooden spoons and a cord as a young child and teenager. She also described this in two journal entries written in her adolescence that she provided to The Post. Several others inside and outside the family also said Ray used physical discipline. In a journal entry dated Dec. 17, 1992, Ziebart, then 12, described one conflict where her father “grab(b)ed my head and shook me hard.” “I have been having ear ac[h]es and I had one today and when he did that it hurt me bad, inside & out,” she wrote. “I love him so much and [he] shouldn’t have hurt me.”

Ray responded: “We used legal and loving spanking. Period.”

I love the “legal” part. Just in case anyone questions the idea that assault and battery is not “loving,” he’s still got legality of it covered.

And this all gives a horrible twist to another part of the story:

Ray often serves as an expert witness in family disputes, typically where one parent wishes to home-school the children and the other doesn’t. Ray listed about 90 cases where he testified (or was prepared to when the case settled) on a document submitted to a Washington state court in 2022.

On the plus side, he doesn’t seem to have tried to run anyone over in a parking lot.