Things I’ve learned from running OTIS

Note: if you are a prospective OTIS student, read the syllabus instead. More useful, less bragging.

In the unlikely event that I’m a social gathering like a party or family gathering, people will sometimes ask me about my teaching. Invariably they ask, “so do you do like 1:1 meetings or group lessons?”. Then I have to explain, no, I have 400 students, there are no synchronous meetings at all. The core of the program is literally a Python web server that serves PDF files.

Then it sounds less impressive. I guess when people hear I’m a teacher, they expect me to teach classes, and it’s a bit embarrassing to explain that I’m not a teacher in that sense anymore.

But the purpose of OTIS isn’t to make Evan sound cool at parties; the purpose of OTIS to be effective for the students. So this is going to be a post about some of the ways OTIS has evolved, from a traditional lecture-homework class with 5 students to today.

Summary

When blogging, I like to write about what was most surprising. So I would say the things that surprised me the most about running OTIS have been:

1. A lot of the necessary work turns out to be routine and unglamorous, like answering emails or writing up solutions.
2. Lectures suck even more than I thought they did…
3. … but only if you write well.
4. You can empower self-motivated students by granting them a lot of autonomy.
5. Engagement matters.
6. Infrastructure matters.

Now I’ll talk more about each of these.

1. Grunt work

When I was young I saw a post from Richard Rusczyk’s blog about his answer to the question “what do you do for a living?”:

Old answer: “Create materials and opportunities for students interested in mathematics.”

New answer: “Write emails.”

Now that I run OTIS, I know what he means. (See my recent snapshot of what OTIS work looked like one Monday.) Chores like answering emails or writing up solutions aren’t exactly flashy, but they are going to make up a lot of the day-to-day work that you do. Reminds me of light bulbs and batteries.

2. Lectures suck

I don’t believe in lectures. I was lukewarm when I started teaching, and now I hate them.

Early in my teaching career, I wrote a blog post saying that “you can’t excise the in-person meeting altogether, but the dirty secret is that the classtime isn’t the core component”. It turns out I was wrong. You can excise the in-person meeting altogether. OTIS has done that for years now and it works great.

There are two places I can think of where lectures were helpful:

• Lectures give the student an opportunity to ask questions and have them answered in realtime, whereas writing does not. However, starting around 2019 this stopped becoming an issue for me with the birth of the OTIS Discord server, which now has about 800 members. When you have a question, you can just post it in Discord and likely Evan or some other student will answer it within an hour.
• The other is motivation and engagement. Talking to a person feels better than reading text. A median high school student does not have the interest or motivation to self-study, so mandatory lectures can be used to force them to absorb at least some material by osmosis.¹ Of course, this benefit decreases the more motivation your students have.

In basically every other aspect, I have consistently found written text to be superior to traditional lectures. Examples:

• Written text can be read by the student at their own pace, rather than at a pace controlled by the instructor’s delivery. (This is even worse in group lessons, since now all students move through every part of the lecture at the same pace.)
• There’s no time limit. Lectures are confined to a schedule. A student reading text on their own time is much more flexible.
• It’s much easier on the student’s memory; they don’t have to frantically copy down stuff from a chalkboard while listening to an instructor talk.
• And writing scales — once I have written something up perfectly, I can reuse it on every current and future student for no additional cost: no need to deliver the same performance to each batch of students each year.
• When students have feedback on the way material can be improved, I can implement it right away instead of having to remember it for the next performance.

You’ll notice that I’ve started using words like “delivery” or “performance”, and that’s deliberate. To do lectures well, you need to rehearse things like not saying “um”, choosing really flashy or shiny topics, managing blackboard space, keeping an audience engaged, and cutting out content to streamline the presentation (whereas in writing, you can e.g. put digressions in footnotes). I started feeling uncomfortable with the realization I was actively making my classes less effective in an effort to make them more popular.²

3. Write everything out.

Of course, that only works if you write well. When writing olympiad content, good taste in choosing problems and choosing themes is important, and the units are one of the biggest strengths of OTIS.

But it’s just as important to actually @#%^ing write things out and not do this:

I urge fellow teachers of math to be cynical of phrases like “left as exercise”, “easy to see”, “the reader can check that”, etc. If your student reads this on something they genuinely cannot figure out how to prove, how many of them will swallow their pride and ask? You aren’t working with students that need whip-cracking; if anything, most motivated students are too unwilling to give up. If you don’t have the time right now, can I at least persuade you to use “TO BE ADDED; ask if you need help” instead of “exercise for reader”?

Consider solution-writing. AoPS solutions are often densely kludged and missing diagrams, and official contest packets can be skimpy too. (Though better than nothing — how many college professors actually write out full solutions to their own problem sets?) I suspect there’s a mindset that writing the exposition is the main work, and the solutions are an afterthought. I encourage you to fight the herd; treat solution-writing with the respect it deserves.

Remember, writing scales — if you write something perfectly once, it benefits all current and future students. That’s why so much effort is put into polishing and cleaning solutions packets for official USA contests, featuring alternate solutions, full-color diagrams, remarks on generalizations or counterexamples showing conditions are needed, etc. Even though solution-writing is one of the unglamorous time-sinks I mentioned, I insist it really, really matters for the students.³

Student feedback is a good way to pinpoint the weakest points. Whenever a student finds something I wrote unclear, I prefer to immediately rewrite it while replying to them. (You’ll see this in several places in the time log.) Indeed, if I don’t edit now, someone else will eventually ask the same question.

4. You can empower motivated students, and it’s great.

If you teach normal K-12 classes, forcing unmotivated students to work is tough. With olympiad students, this isn’t a problem anymore (when people ask me how COVID affected OTIS, I reply that it tripled student productivity). So one OTIS trend has been to grant more freedom to students.

Examples of this:

• OTIS doesn’t have deadlines, and gives you complete freedom of which units you want to work on. The fact you can pick which topics you want to cover is commonly cited as one of the strengths of OTIS. Because, the key thing here is, olympiad students are smart: these students know themselves much better than I do. I trust in their judgment to decide what’s most useful for them to learn next, over my own ability to prescribe a fixed course.

• One of the biggest milestone changes to OTIS in 2018 was the introduction of point-based problem sets; the unit has a bunch of problems, but you pick which ones you want to solve. When people ask me for advice on which ones to do, I just tell them to do whichever ones look the most fun. The linked post has more explanation about why I think this worked so well.

• There’s a streamlined process now for getting hints to problem sets, or downloading full solutions⁴ if you’re completely stumped. These don’t require permission from Evan or even any human interaction, so students don’t have to feel guilty about asking a real person and admitting they don’t want to work on a problem anymore. You just trust the student to take care of themselves. (If anything, I think more of my students are too reluctant to give up on a problem than give up too quickly. But it’s a spectrum.)

I can’t find the exact quote, but I remember Richard Rusczyk saying somewhere that when mentoring great students, your role goes from less of a traditional pass-on-knowledge teacher to a guide. I like this sentiment. I don’t like to be an elevated authority figure; it’s just not my style.

5. Engagement matters.

Maybe the last section is an exaggeration; it does pay to increase motivation. I just think there are better methods than being authoritative or, heaven forbid, scheduling mandatory lectures at fixed times.

Examples of ways which OTIS tries to do this:

• The point-based problem sets were a big part of this. It was really important students counted the number of problems they solved rather than the number of problems they had left to do.

• Then I had the OTIS website automatically track the total number of points earned across all submitted units, and display a level⁵ for the student. That’s a lot of fun for the kids; they see the level go up as they progress.

• Artwork. Yes, really. We have cover art now.

• Finally, OTIS uses a “buffet unlock system”: you get some PDFs to start, and you don’t unlock new ones until you finish those you already have. This is important because it creates an incentive to keep progressing. (In the past when people had access to everything, there was a lot of complacency of the form “oh, I can always work on this later”, and then never actually getting to work on it.)

6. Infrastructure matters.

I kept talking earlier about how writing scales. You know what else scales? Code.

I really like Chris Petey’s quote:

The most competent people, with weak processes, will screw up.

That’s a lot of my experience with running OTIS too. The OTIS infrastructure has a lot of weird quirks and technical debt, but it’s there. Parts of the tech stack include:

• The central OTIS-WEB server, handling user authentication, payments via Stripe, problem set submissions, the database of prewritten hints, and so on.
• Evan’s problem database, named VON, which provides a central source of truth for statements and solutions.
• The OTIS materials repo, which tracks the unit source files and automatically generates solutions packets and other assets.
• The venueQ module in my dotfiles, which provides a streamlined staff interface to the OTIS-WEB API.
• The Discord server for all social stuff, including a problems bot.

(Yes, it’s all Python. Yes, I wrote it all myself.)

Without the code, OTIS would simply not exist. So much credit of OTIS is due to the smoothness of the operational side, which has nothing to do with the academic side. And this infrastructure is always being improved; the main OTIS-WEB repository gets almost daily commits⁶.

For example, it wasn’t until November 2022 that the OTIS database had a URL field, making it possible to jump directly to the AoPS thread in one click. In theory, the student could manually find the problem themselves from the AoPS contest index. But in practice, making this process frictionless matters, and now I can’t imagine pre-URL life. All these little things add up, scaled by the number of students.

Pay attention to plumbing and wiring. It matters at least as much as the content.

Closing thoughts

… I feel like I basically wrote an ad for OTIS. This was meant as a post about how OTIS improved over time, but even then is kind of over-the-top.

Sorry. If you’re a fellow teacher of superstar students, let me try to summarize the actually useful advice: engage and empower your students, and focus on mundane things like answering emails, polishing solutions, and building infrastructure. Hope that helps.