Endearments like this don't bother me much in normal conversation when they are hardly more than just punctuation for some people. I think there can be trouble if there's any question of presumed status though - for example, one time I did get cross about something similar was when I was making a complaint to my bank after some mess-up of theirs and the person on the phone called me by my first name. Given the nature of the conversation up to that point, the false friendliness just came over as disrespect. I guess this is exactly what the OP felt too.
