When Will My Kitten Calm Down?

I have a kitten that I adopted from the kitty town shelter. She is about 3 months old now. We took her home with us and she has become very attached to me, but then my husband went out of town for about 2 weeks on business trips, so she was alone in our house all by herself for most of those days. At first it wasn’t too bad because we had other cats who would check on her every once in awhile, but now that he’s back I’m having trouble making sure she stays calm around him! She does not like men at all (that is why we got him). However whenever him or his friends come over they go into their separate rooms and stay there until they leave which is usually after dinner time (which can be anytime between 11:00 pm-8:00 am). So my question is how do i make sure my cat will let them hang out without attacking? How do you get your cat to trust someone you’ve never met before? Do you think it’ll work eventually? Also what kind of litter should i use for this situation since its an outside only area??? Thank You!!!!