Against Perfect Scores

One of the pieces of advice I constantly give to young students preparing for math contests is that they should probably do harder problems. But perhaps I don’t preach this zealously enough for them to listen, so here’s a concrete reason (with actual math!) why I give this advice.

1. The AIME and USAMO

In the USA many students who seriously prepare for math contests eventually qualify for an exam called the AIME (American Invitational Math Exam). This is a 3-hour exam with 15 short-answer problems; the median score is maybe about 5 problems.

Correctly solving maybe 10 of the problems qualifies for the much more difficult USAMO. This national olympiad is much more daunting, with six proof-based problems given over nine hours. It is not uncommon for olympiad contestants to not solve a single problem (this certainly happened to me a fair share of times!).

You’ll notice the stark difference in the scale of these contests (Tanya Khovanova has a longer complaint about this here). For students who are qualifying for USAMO for the first time, the olympiad is terrifying: I certainly remember the first time I took the olympiad with a super lofty goal of solving any problem.

Now, my personal opinion is that the difference between AIME and USAMO is generally exaggerated, and less drastic than appearances suggest. But even then, the psychological fear is still there — so what do you think happens to this demographic of students?

Answer: they don’t move on from AIME training. They think, “oh, the USAMO is too hard, I can only solve 10 problems on the AIME so I should stick to solving hard problems on the AIME until I can comfortably solve most of them”. So they keep on working through old AIME papers.

This is a bad idea.

2. Perfect Scores

To understand why this is a bad idea, let’s ask the following question: how good to you have to be to consistently get a perfect score on the AIME?

Consider first a student averages a score of on the AIME, which is a fairly comfortable qualifying score. For illustration, let’s crudely simplify and assume that on a 15-question exam, he has a independent probability of getting each question right. Then the chance he sweeps the AIME is

This is pretty low, which makes sense: and on the AIME feel like quite different scores.

Now suppose we bump that up to averaging problems on the AIME, which is almost certainly enough to qualify for the USAMO. This time, the chance of sweeping is

This should feel kind of low to you as well. So if you consistently solve of problems in training, your chance at netting a perfect score is still dismal, even though on average you’re only three problems away.

Well, that’s annoying, so let’s push this as far as we can: consider a student who’s averaging problems (thus, success), id est a near-perfect score. Then the probability of getting a perfect score

Which is\dots just over .

At which point you throw up your hands and say, what more could you ask for? I’m already averaging one less than a perfect score, and I still don’t have a good chance of acing the exam? This should feel very unfair: on average you’re only one problem away from full marks, and yet doing one problem better than normal is still a splotchy hit-or-miss.

3. Some Combinatorics

Those of you who either know statistics / combinatorics might be able to see what’s going on now. The problem is that

for small . That is, if your accuracy is even a little away from perfect, that difference gets amplified by a factor of against you.

Below is a nice chart that shows you, based on this oversimplified naïve model, how likely you are to do a little better than your average.

Even if you’re not aiming for that lofty perfect score, we see the same repulsion effect: it’s quite hard to do even a little better than average. If you get an average score of , the probability of getting looks to be about . As for the chances are even more dismal. In fact, merely staying afloat (getting at least your average score) isn’t a comfortable proposition.

And this is in my simplified model of “independent events”. Those of you who actually take the AIME know just how costly small arithmetic errors are, and just how steep the difficulty curve on this exam is.

All of this goes to show: to reliably and consistently ace the AIME, it’s not enough to be able to do 95% of AIME problems (which is already quite a feat). You almost need to be able to solve AIME problems in your sleep. On any given AIME some people will get luckier than others, but coming out with a perfect score every time is a huge undertaking.

4. 90% Confidence?

By the way, did I ever mention that it’s really hard to be 90% confident in something? In most contexts, 90% is a really big number.

If you don’t know what I’m talking about:

take three or four minutes and do the following quiz.

This is also the first page of this worksheet. The idea of this quiz is to give you a sense of just how high 90% is. To do this, you are asked 10 numerical questions and must provide an interval which you think the answer lies within with probability 90%. (So ideally, you would get exactly 9 intervals correct.)

As a hint: almost everyone is overconfident. Second hint: almost everyone is overconfident even after being told that their intervals should be embarrassingly wide. Third hint: I just tried this again and got a low score.

(For more fun of this form: calibration game.)

5. Practice

So what do you do if you really want to get a perfect score on the AIME?

Well, first of all, my advice is that you have better things to do (like USAMO). But even if you are unshakeable on your desire to get a 15, my advice still remains the same: do some USAMO problems.

Why? The reason is that going from average to average means going from 95% accuracy to 99% accuracy, as I’ve discussed above.

So what you don’t want to do is keep doing AIME problems. You are not using your time well if you get 95% accuracy in training. I’m well on record saying that you learn the most from problems that are just a little above your ability level, and massing AIME problems is basically the exact opposite of that. You’d maybe only run into a problem you couldn’t solve once every 10 or 20 or 30 problems. That’s just grossly inefficient.

The way out of this is to do harder problems, and that’s why I explicitly suggest people start working on USAMO problems even before they’re 90% confident they will qualify for it. At the very least, you certainly won’t be bored.