I know what you mean...
I seem to be affected by this too - sometimes when I walk by some people, I can sense great anger or negative energy from them, and it is almost like I am drowning in their thoughts - it's not a pleasant experience and I do get worked up by it and it takes me a while to calm down and get over the thoughts and energy...
There are protection exercises - where you can imagine yourself surrounded by a sphere of light which can protect your soul from other people's energies...
Here are some links:
http://www.bellaonline.com/articles/art32977.asp
http://home.vicnet.net.au/~johnf/defence.htm