What am I to do?

Today at tee ball Magnus was picked on by a bunch of boys on his team. They kept on trying to pop his hat off of his head then trying to twist his head around and there I watch as my son does nothing. At one point a boy threw his hat on the ground and my son did nothing, but retrieve the hat. I was so upset at him for not doing anything about it. After practice I asked him how he felt about his hat being tossed around and he say he didn't like it a little bit. Oh my boy! There I got all emotional and then I realize that I played a part in it too. I could have gone over to where they were practicing and my presence alone probably would've deterred the kids from doing it. Then I felt really guilty like the worse mom in the world. It is so hard. I had debated about going to there to tell the kids to stop, but I didn't know if I should barge in or let him handle the situation himself. I tried to play it cool and obviously that didn't work out so now I will have to be the overbearing parent that everyone thinks is annoying, but oh well. This is my kid we are talking about and I am not going to let him get bullied. Please pray for my emotions, that I won't rip the heads of the kids who are giving my son a hard time. Magnus is just so different than the rest of the pack. He doesn't mess around and is really quiet. He likes to watch the other kids and I think that somehow that sends a message to the kids that they can mess with him.