This is why men keep their guards up when dating