There are some interesting moral issues related to the use of expressions that are perfectly ordinary and inoffensive in most situations but are used offensively within a small subset of the population, particularly when there are some among those on the receiving end of such expressions who don't know of the ordinary, inoffensive use of the term in question. It's usually good to show moral deference to the ignorant, if we haven't been in their position of ignorance, giving them the benefit of the doubt. But the ignorant in these cases include both (a) those who use the expression without knowing or the offensive connotation that it has in certain contexts and (b) those offended but its ordinary usage because they don't know about anything other than its offensive use. At the same time, there's always the questions of (c) whether those in (a) ought to have been more aware of what offends people and (d) whether those in (b) ought to have be willing to throw out such serious moral charges based on an ignorance that many might not easily excuse.
I've defended the use of such expressions in many contexts, emphasizing (a) and (d) above while perhaps too easily dismissing (b) and (c), or at least not explicitly laying out the reasoning for why I tend to favor (a) and (d) as more decisive in these kinds of cases. One example that came up in my post was the old expression "call a spade a spade". This one actually goes back to Plutarch in the second century, according to the Oxford English Dictionary, although he used a different metaphor that was later mistranslated by Erasmus in 1542. (It's not generally accessible online except with a password to get through a university firewall, or I'd link to it.)
When I was talking about these cases with my friend and colleague Chuck, who occasionally comments here, he decided to go check the OED to get the history of the expression. He noticed a particularly funny quote that the OED used to exemplify "call a spade a spade".
1647TRAPP Marrow Gd. Authors in Comm. Ep. 641 Gods people shall not spare to call a spade a spade, a niggard a niggard.
Those who have followed the recent history of offense over normally-inoffensive terms will remember that the black mayor of the District of Columbia fired one of his white aides for using the term 'niggardly', a word that only sounds like a racial epithet if you aren't listening very carefully. Even the NAACP chair, Julian Bond, thought it was crazy to criticize someone for using that word. But I suppose we've now got solid proof that 'niggard' does refer to black people, since Trapp in 1647 used it in parallel with "call a spade a spade". Or does this show that "call a spade a spade" is tied to offensive language because its connection with niggards goes back at least to 1647?