Oh, so much to say. But first, this.
The following article does not contain New-Age Guilt.
Do your thought-patterns, emotions, and beliefs have an impact on the physical body? Yes. Yes they do. 
Candace Pert, Ph.D., wrote in her book, Molecules of Emotion, about how our bodies and minds function together as parts of an interconnected system. Fascinating stuff, that.
Louise Hay’s book, You Can Heal Your Life, lists probable thought-patterns related to physical ailments and suggests new though-patterns for each. For instance, she lists hip issues as a fear of moving forward. 
You can use this as a starting point, perhaps by diving into some self-discovery through journaling or working with a therapist or coach.
And… ever the fan of the paradox…sometimes stuff happens and we cannot access the why, because of our limited view of reality in this dimension. It may be due to group thought-patterns, or beliefs inherited from family, or, or, or. 
But we can choose how we perceive our experiences, what thoughts and feelings we decide to entertain in every moment. We can engage our internal world with love, compassion, and acceptance as we decode the thought-patterns our body is urging us to shift within ourselves.

How can I clear the effects of being around negative people? It seems to really impact me.

Think of emotional frequencies as radio stations on a dial. It sounds like you just allow yourself to change stations. Don’t do that. 
I’m only half-kidding because I know how hard it can be. But it truly is an inside job. It’s all about managing your own frequency and energetic boundaries.
Remember, you don’t absorb the emotions of others, you match your frequency to theirs. You basically recreate those same negative emotions for yourself, then you feel bad too. Not helpful for anyone.
If you find yourself in a puddle after hanging out in a fear-based vibe, try any or all of these to clear and raise your frequency to love or above:
Breathe in and imagine your energy body expanding. 
Come back to the present moment. This will help to clear the emotions that you duplicated for yourself that have nothing to do with you.

Focus on your heart chakra and bring to mind something or someone you love and appreciate. You can do this anywhere, even when you’re in the thick of it.
Go outside. Take a walk and ground in nature if you can. As you ground, breathe deeply and as you exhale, imagine letting all of it fall away. Nature loves to recycle our messes. Remember to give gratitude and loving energy back.
I’m a huge fan of Epsom salt baths. They clear and reset the energy body.
If all else fails, watch Brian Regan’s Refrigerator Salesman stand-up bit on YouTube. This may actually be the most effective.