(Ignoring OotP) I know he's grumpy and gruff, but I have to wonder if *all* students hate him so much. Wouldn't there be at least a few sensitive ones who would smile at him now and then? Pat Mrs. Norris? Say hello in the beginning of the year? I'd like to think so. *I* would. (Yes, even when I was a kid I tended to try to cheer ever-grumpy people up. No, it didn't usually work, but even if there weren't outward signs of success, I had hoped that it makes the person feel a little better inside.)
I say 'ignoring OotP' because ... Because. I hate calling "plot hole!" and the like on JKR, but. Well. I know Filch says he wants to torture the kids, but if he really did want that, then there's no way Dumbledore would keep him around. If he's still there in the next book, my eyebrows will lift very high and I'll throw something in the direction of England. (Why would Dumbledore keep him around when he sided so openly with Umbridge?)
I keep Filch in my 'Men who really only need a hug to improve' box. He can keep Snape company in there.