Fear and Loathing in the Moodle Community

Moodle News responded to our recent coverage regarding the platform’s declining market share by pushing back hard with an article that simultaneously insinuates bias on our part and attempts to use our numbers to draw the opposite conclusions from the ones that we have put forward. This presented us with something of a dilemma. As professional analysts, we try very hard to be undefensive when we are critiqued. If people think we are biased, they are entitled to their opinions. If people question our data or analysis, we try to respond only if we think their critique of us has merit. If we don’t think it does, we generally let our original work speak for itself unless we have a specific reason to do otherwise. So, from this perspective, our default editorial position is to let Moodle News have their say and leave it alone. I will add that our overall impression of Moodle News has been that it is generally a fair and thoughtful outlet. We have no particular desire to pick a fight with them.

On the other hand, we are not just analysts. One of the more common compliments we get from people who trust our work is that they believe it is animated by a concern for improving education. I take these comments to mean more than just that we “care” in some abstract sense. Rather, they seem to be saying that our choices of what stories we cover and how we cover them are animated by our desire for our analysis to be a tool for educators to help improve education. From that perspective, it’s hard for me personally to let the Moodle News story go unanswered. I care about what happens to Moodle, not so much because I care about Moodle in and of itself but because I care about the good that the platform and the community do for education across the globe.

Sometimes when we poke at a group because of a problem, it’s partly because we want to call their attention to it in the hopes that they fix it. If we poke harder, it may be because we’re not convinced that they’re paying attention to the dangers that we see. (See, for example, Phil’s recent Unizin coverage.)

Beyond the Moodle News piece and the occasional (but energetic) challenges we get from Moodle advocates when we present our numbers at conferences, the case for poking here is bolstered by Martin Dougiamas’ periodic public questioning of our analysis, including his comment on that aforementioned last post. The bulk of that comment was as follows:

I get that you are a fan of certain companies and that is fine, but I don’t understand the highly negative and uninformed spin from you lately. Look at the title of this article! These feel like intentional attacks, to be honest, and I have to wonder why.

The fact is your entire article here is based on the false supposition that our business model is a) static and b) based entirely on higher ed. However, we already have a number of new and exciting initiatives that you clearly don’t know about (MoodleCloud, MoodleNet, LearnMoodle, MoodleServices as well as new things not announced yet) that are supporting our current and future growth.

Sustainability of Open Source and all Open initiatives is something we care about deeply and is part of everything we do.

Again, readers have to come to their own conclusions regarding the quality of and motivations for our posts. From our perspective, we tend to write negative headlines when we see negative news. We tend to get more negative in our tone when we think the people we are trying to reach are not hearing us (or are not honest, though that is not an accusation that I am making here).

As somebody who has been very directly involved with and committed to open source projects myself, I have learned that the very passion which drives participation can also cause advocates to dismiss any bad news as “fake news.” This can be fatal to a project. Moodle has massive market share, a fresh infusion of cash, and plenty of talent. There is time to address any challenges that the project faces. But only if those challenges are faced.

I have decided to take one more run at this topic because I would like to see the Moodle community succeed, and in order to do that, I believe it will have to grapple with challenges that I don’t see evidence that it is fully grappling with yet.

The Passion Play of Open Source in Crisis

While the role I have chosen for myself in the ed tech ecosystem has required me to be ecumenical in recent years, I had previously been an active participant in two different open source LMS projects. The first was a system called dotLRN, which came into existence around 2000 and seems to have died around 2010. (The web site is still up, but nobody appears to be home.) From a functional perspective, dotLRN was fantastic. In retrospect, it was a decade ahead of its competition in a number of ways. But its technology stack was quirky. It was written in a programming language called TCL—which stands for “tool command language” but is referred to by its proponents as “tickle”—and ran on top of an early open source application server called AOLServer. I was told by people I trusted that these were perfectly valid technical choices that had distinct advantages over contemporary alternatives. As a newbie to software development and a non-engineer, I had no reason to doubt those assessments.

But while I didn’t know much about technology, I did know how to listen. Over time, it became clear to me that attracting new developers and winning the confidence of university IT departments would be hard. Nobody seemed to think that learning a programming language called “tickle” and an application server called “AOLServer” seemed like a path to a career on the cutting edge. I grew more confident in that assessment as it became clear that adoption growth of the platform had hit a wall and it wasn’t even getting considered as an option in most cases. I raised this concern with the community, but many of the engineers continued to insist that the superiority of their choices would win in the end. And they could point to all the strengths of their platform relative to the competition as evidence for their case.

The problem came to a head one day for me when I made a discovery about one of those strengths. There was a situation—I’ve long since forgotten the details—where the dotLRN developers decided to turn one of the capabilities of the platform into a web service. I asked, “How hard was that to do?” The answer? “Trivial. It’s just configuration. We can turn pretty much any API call into a web service with the flip of a switch.”

I was stunned. This seemed like the way out! Let the dotLRN core continue to be developed by a limited number of specialized engineers, similarly to how web servers like Apache were being written in C by a small number of hard-core specialized developers. Develop a new front end using a more popular language, which at that time would have likely been PHP, Python, or Java. Use web services to communicate between the two. This was before REST and JSON had really hit the scene, so they would have been XML-based web services. But the point was, nobody would have to learn TCL. It would change everything, I thought.

The dotLRN developer community was less enthusiastic. They believed their development approach was technically superior. Why should they pander to the least common denominator by promoting an awful, inelegant language like PHP? Besides, they had a huge installation in Brazil. Things were looking up, they said.

It turns out that many open source communities, when faced with declining adoption that is forcing consideration of unpalatable choices, become some version of David Hasslehoff. “I may be the butt of jokes in the US, but I’m huge in Germany!” That is a bad place to be. Before you know it, the only things people remember you for are your deeply unfortunate “drunken stupor” YouTube video and your cameo in the SpongeBob movie.

Could a switch to web services have saved dotLRN? I don’t know. It would have been tough no matter what. But the point is that open source communities are often driven by the passion of the participants. While it was exactly that commitment to the mission that attracted me to dotLRN at a time when the proprietary alternatives made me very queasy, it was that very same emotion that blinded some of the most passionate and committed community members to problems that turned out to be existential.

When I left the dotLRN community, it was in the process of tearing itself apart. There was a lot of internal debate about whether anything needed to be done for the health of the project and, if so, what. One particularly bright and charismatic young leader in the community became offended that there was resistance to the direction that he (and his company) wanted to take dotLRN. So he left. (I hear he has since gone on to do great work under the auspices of the Mozilla community.) There was a sense among the remaining members that their suffering was externally inflicted. How could the world fail to appreciate the wonderful things that they had built and shared for everyone’s benefit? They were the victims, sacrificed by the unappreciative mob as thanks for their self-sacrificing effort. And now....

I have seen similar challenges, albeit to a much lesser degree, in the Sakai community. By and large, that community has a more realistic grasp of where their challenges are and what their niche is. They also appear to be fairly stable at the moment.

But passion can blind the Sakai community members from time to time too. (“We’re big in Spain!”) I can remember a time not too many years ago when people in that community—smart people who I respect and whose livelihoods depended on the health of the platform—who thought everything was just fine. One in particular told me that all Sakai needed was a refresh of the grade book and the test engine. Otherwise, everything was just great. (I’m not sure whether this was before or after Sakai finally fixed a fundamental problem where the platform broke the browser back button, but if it was after, it wasn’t long after.)

That guy is now the CTO for a company that is not Sakai-focused.

More recently, the last time I was at an Apereo conference, I attended a session whose abstract said the presenters would talk about how they welcome RFP processes and have kept Sakai competitive at their schools. I was curious. What followed was a litany of the presenters raising every imaginable obstacle to choosing Sakai over competitors, including many reasonable ones, and off-handledly dismissed them one by one. Perhaps inevitably, they came around to Phil’s squid diagram. They clearly didn’t know either that I was connected to the graph or that I had previously served on the Sakai Foundation Board of Directors. I didn’t say anything; I hadn’t gone to the session to ambush the poor guys. But others in the room knew who I was. All eyes turned to me, and several people pointed to me. The guy presenting in the moment rehearsed his arguments.

”That graph is based on old data, right?”

No, you just have an old version. The latest version is up on the blog, and it doesn’t look any better.

”But this is just a small sample.”

About 90% of US and Canadian institutions.

You get the idea.

No organization can remain healthy and effective if it doesn’t balance its passion with a healthy dose of skepticism and self-reflection.

Which brings us to the Moodle News piece.

Getting the Numbers Right

There appear to be three sources of misunderstanding in the article in question. The first is not understanding our point about the intersecting trends of the collapse of new implementations (meaning very few new schools choosing to adopt Moodle in a particular year) with an increase in decommissions (meaning more schools replacing Moodle with another LMS). In North American higher education, we are already seeing a decline in total Moodle institutions of 45 – 50 (depending on when you measure). This is roughly equal to the 1% rounding number that we mentioned in the post (i.e., 1% of 4,500+ institutions in North America dataset).

The Moodle News author is correct in noting that for the horse race view, Blackboard and Moodle are closer than ever when measured by percentage of institutions using as primary system. However, this is mainly because Blackboard’s market share has collapsed much faster than Moodle’s. Arguing that this shows Moodle is strong is a little bit like arguing that MySpace is strong because it has surpassed Napster’s market share (though I will grant that a slow decline is better than a fast one).

The second problem is in not understanding the difference between new implementations and installed base. The chart I referenced in my last Moodle post measures the percentage of new implementations in each year moving towards Moodle – this is where (at the time of article) 0% of 2017 new implementations went to Moodle, 55% went to Canvas, and 44%. (To be fair, we’ve detected a few new implementations since that last piece; we will include them in our next periodic update of the graphs.)

The author then asks:

All of which leaves us to wonder: How can an LMS have no growth while its close competitors show increases of 55% and 44% (for D2L, now with a 15% market share), and end the year with its market share unchanged?

Market share is based on installed base—the percentage of institutions having a particular LMS as its primary system as measured in a point of time. To get this number, we add new implementations, subtract decommissions, account for the changing denominator in the total number of institutions (there are a lot of consolidation and closings happening in the US), and round to the nearest whole percentage.

So the answer to the author’s question is that Moodle has lost 45 - 50 net institutions in North American higher education. In other words, roughly equal to the 1% rounding number.

The third problem is not understanding the difference between primary institutional adoptions, which is the measure that we use in our analysis, and the measure of the number of registered sites in Moodle.org’s stats. We don’t, for example, count the sites that may be self-run by an individual professor or department and that may be registered with Moodle.org. The latter may be important to the community from a mission perspective, but those adoptions do not impact the financial resources available for development of the platform. You can’t combine the two numbers.

Speaking of which, the situation looks even worse for Moodle the closer we get to counting things that directly translate into revenues. The numbers that we track in many of our charts and graphs are the numbers of institutions that adopt different platforms. But the numbers that really matter for the financial health with most of these platforms is the number of students, because institutions pay their vendors, including the Moodle Partners that bankroll Moodle HQ, by the enrollment. For example, Southern New Hampshire University (which is moving to D2L) and Glendale Career College (which is on Moodle) each count as one institutional customer. But SNHU has over 100,000 students, while GCC has 300.

When measuring by numbers of students, Moodle is fourth in the US and Canada, behind Blackboard Learn, Canvas, and Brightspace—and these numbers do not include future losses from the University of Minnesota and other schools that have chosen to migrate off of Moodle in the near future. Note the numbers in the right-most column of this chart:

If, as our data show, the Moodle market share in US and Canada is much smaller when measured by numbers of students versus institutions, even as the number of Moodle institutions is shrinking, that is a bad picture for Moodle HQ’s financial health. If you care about Moodle, then you should be worried about these numbers.

A Healthy Skepticism of Skepticism

Is it possible that we at e-Literate are biased against Moodle? Of course it is. While we try our best to be objective, we are humans, and humans can be biased. The Moodle News author spends a fair bit of energy insinuating that we have a preference for other platforms. I’m not going to address that question, partly because I’m not the best judge of my own biases and partly because it’s not the question that Moodle advocates should be asking. Rather, the main question they should be worried about and should be investigating with as much absence of bias as they can muster, is the following:

Is e-Literate’s analysis true?

Could we be wrong? Of course we could. I very much doubt that we are off by much in the US and Canada, where we our data are the strongest. The margin of error increases as we get to parts of the world where our coverage is less complete (or, in some cases, non-existent). Those areas should be clear—and, in fact, are referenced in the Moodle News article—because we quantify the data fidelity in our regional analyses. If Moodle is huge in Germany, or China, or Myanmar, we might not be picking that up in our data.

There is one person who may well have better data than we have, and Moodle advocates have reason to believe that he is not biased against Moodle. His name is Martin Dougiamas. He argued in the comment quoted at the top of the post that we don’t really understand or have visibility into the business model of Moodle Pty. If that is true, then we would like to be enlightened. And the Moodle community should want that too, for peace of mind.

Moodle Pty could provide the community with two kinds of information that would help its advocates understand how much they should be worried (or not). First, Moodle Partners likely give Moodle Pty counts of customers by institution and numbers of students. (I don’t know this for certain, but I’m not sure how Moodle Pty could verify that they are being properly paid by the partner without this information.) The company could publish this information, aggregated by country so that individual customers and partners have some degree of anonymity while still giving the community a sense of the inputs that fund Moodle development. A second disclosure that Moodle Pty could offer is direct revenue numbers that are transparent enough for community members and other interested parties to independently verify the company’s financial health. While Moodle Pty is not legally obligated to provide any of these numbers, there are disclosure models in both non-profit foundations and for-profit companies that the company could choose to follow. At the moment, the status of Moodle Pty as a private corporation shields it from transparency requirements of either non-profit foundations or publicly traded corporations. But that doesn’t mean that the company, as the main engine of sustainability for a huge open source project that, despite current growth challenges, remains by far the world’s most widely adopted academic LMS, couldn’t or shouldn’t choose to be transparent about numbers that are critical to the project’s sustainability.

If Moodle News really wants to check our numbers, then they should be asking Moodle Pty for adoption numbers of Moodle Partner-supported installations by institution, headcount, and/or revenues. And their biggest concern, as Moodle advocates, should not be whether some US analysts are talking smack about their favorite platform. It should be about how healthy and sustainable that platform truly is.

The post Fear and Loathing in the Moodle Community appeared first on e-Literate.