What if I love my boyfriend but not his friends?

me and my boyfriend have been on and off now for going on 2 years. i’m 18 and he just recently turned 21. he can be the best boyfriend ever sometimes and SOMETIMES we seem SO perfect together. then theres a good many times, like the weekends when he just seems to show his friends are more important and he’ll dump me if i don’t go along with all he wishes to do like go stand around in a yard and watch him and them drink them drive him home. sometimes he’ll cuss at me and make me cry and it really hurts my feelings i know you’re thinkin get rid of that loser. but i’ve tried several times. we’ve spend 2 seperate months apart and i’m more miserable without him than i am with him. i want to be with him and theres no changing my mind. but, i just really can’t make myself like all of his friends because of situations in the past its like they try and ruin us and he just doesn’t see it. all my girlfriends either date or hang out with the same crowd and we all live in a small town so theres really no way around it. i’ve tried dating other guys, and other guys from other towns but it just doesn’t work. please give me some suggestions of how i can have fun with him also but not including me drinkin and driving just to fit in cause i don’t do that. and also how i can somehow manage his friends so our relationship won’t fail again cause it’d kill me to see him with someone else and it has happened before already. i know deep down he’s the only one i want.. advice please!