After my opponent served, his hat came off as he came into
the net. I was distracted and missed my shot. Is this a hindrance, and do
I get to replay the point?
Yes and yes. The rules define a hindrance as, "If a player commits any act which hinders his opponent in making a stroke, then, if this is deliberate, he shall lose the point or if involuntary, the point shall be replayed." Since your opponent's act was involuntary, the point should have been replayed.