Matthew Franck notes that on one of Barack Obama's exam questions from when he was teaching law, he asks whether an equal protection challenge can be brought against a law requiring states to be color-blind. Franck says he knows of lots of people who think the equal protection clause requires states to be color-blind, but he hasn't encountered a serious argument anywhere that such laws violate the equal protection clause. I haven't either, but I don't read law reviews. Still, such an argument isn't hard to imagine, and I think it's actually a sound argument.
The equal protection clause entitles people of all races to equal protection of the laws. The laws therefore need to be able to rely on the distinction between members of one race and members of another if they are to ensure that each race is equally protected by them. Therefore, color-blind laws, which disallow the state from paying attention to race, violate the equal protection clause.
It sounds like a pretty good argument to me. As a policy issue, I don't mind restricting affirmative action in universities to class rather than race, or at least ensuring that the standards aren't lowered as much as they are. There's a significant argument that the way affirmative action is typically practiced in that setting (as opposed to in the workplace, which is a very different matter) seems to me to harm the people it's intended to help, given that admissions officers already go out of their way to promote diversity (so there's no discrimination to combat at that level), and it means accepting people who won't be able to do as well and then will appear less good when they graduate than they would at a lower institution with much higher grades and more time for extracurriculars. There are other negatives too, but that's the one that seems decisive to me. I think it's much better to work at the high school level and below to help kids do better in school, to care more about school, and to think of college as something worth doing.
But I can't see how it could be good to ban affirmative action by not allowing a state to recognize racial distinctions in any way. That sort of law is not just bad policy. It really is unconstitutional because it prevents enforcement of the equal protection clause.