I like the way you handled this situation. I too, am sensitive in the same way and might ruminate too long on a backhanded compliment.

I think it depends on the relationship too. If a person is known for repeating this type of offense, then I would definitely say something, if not then it's better to let it go, say thank you, and move on. 😉