I've always been baffled by the expression "the exception that proves the rule". It never made any sense why an exception could prove a rule. Shouldn't it prove that the rule isn't true?
I once heard an erroneous explanation that it has to do with the older sense of 'prove' as in testing or trying. The exception tests the rule and makes it harder to establish itself. That made a little more sense, but it turns out to be wrong.
The real explanation is much simpler. I was assuming this was supposed to be some absolute rule. The exception that proves the rule does actually confirm the rule, but it confirms it logically rather than empirically. If it's an exception, there's got to be something it's an exception to, i.e. a rule that it goes against. It obviously can't be an exception to an exceptionless or absolute rule. It's got to be an exception to a true generalization. But it can't be an exception without some such rule that it's an exception to. That's how an exception proves a rule.
Now there's still an illegitimate use of this expression, and I see it all the time. Whenever anyone states a rule as an absolute, and someone shows that the rule is false by finding a counterexample, they can't respond by saying that it's the exception that proves the rule. No, it in fact disproves the rule, since the rule was stated as an absolute without exception. So what I was objecting to all along was indeed an illegitimate rhetorical move. It's just that the expression has its origin in a perfectly legitimate point that can be made. It's just not what people usually mean by the expression now.
For more information, see the Wikipedia entry on this. (Yes, unbelievably, it's got its own Wikipedia entry.)