I dunno. I say lots of things to people's faces. Sometimes I say it to someone else first though, just to work it out, or maybe I say it to someone else after, because they didn't listen. Or maybe I said it to them politely and then to someone else less politely.

I put that all in the category of saying to.

Maybe the dividing line is what you do if somebody confronts you and says, "Look, do you have a problem with me?" Do say yes and explain why or do you say, "No, no, everything's fine" and then continue sniping about them behind their back?