"You're so skinny"

"You're so skinny!" <- That's something I've been hearing my whole life, so it never really bothered me. But after a while it gets really annoying. I would never go up to somebody and say "You're so fat!" so why is the opposite okay? Maybe I'm being irrational, but it really bothers me all of a sudden. I know that the people who say it mean well, and they probably even mean it as a compliment, but it just makes me uncomfortable. 

It makes it even worse when they make one of those jokes like "You're so skinny! You need to eat more (insert junk food type here)". It's like I eat plenty, I'm healthy, and I'm in a proper weight range for my height. I've always been a little petite, so get over it. If it's not bothering me, why is it on other people's minds all the time?

I mean, obviously if it was a cause for concern and people were worrying about me, then that would be a different story all together. But that's not the case, so there's really no reason for it. 

And the worst thing you can say is "You're so skinny, I hate you!" I mean I know you don't actually hate me, but it doesn't make sense. Again if someone ever said "You're so fat, I hate you!" that would be obviously rude, so I still don't get why someone would say that about a skinny person. There's kind of a double standard here.

I could rant about this forever, but I'll stop now. Anyways, do you think I'm being unreasonable? Or do you agree that this double standard is unfair?