It's all my fault to him

I've been seeing my "boyfriend" off and on for 2 years now. When I say off and on, I mean off 2 days, back for 5 more and so the pattern continues. I'm a homebody and do not like the social scene very much. I am a member of aa and haven't really been out in the social scene now for four years. He's not into the social scene much either, but sometimes he wants to go out - I'd love to go with him. The problem is that our relationship as I said has been on and off for a reason (reasons). I'm not sure anymore if I'm right or if he's right or somewhere in between. It's not necessarily about being "right" but moreso being "rational". He's turned so many things around on me over the last couple of years, that I have no clue anymore what is and isn't rational or worth arguing about. (almost any discussion turns into an arguement.) I will tell you about some things that have happened in the past which, to me, are not healthy - but now, I question myself since he says it's always my fault.
He says it's my fault when:

1. I break up with him every other day (not literally, but close). The only reason I break up with him is bc he's made me feel so insecure about us by telling me how I always do him wrong, yelling at me, turning things around on me, getting mad about the smallest things, he gets defensive anytime I try to talk about anything - even if I'm not even accusing or even talking about him, my insecurity about his faithfulness, his sneakiness, his controlling behavior at times, the way he cusses at me and tells me what a looney, unstable, selfish, flakey, immature, girlfriend I am, the way he uses the excuse of "me breaking up with him every other day" to keep some girls in his life bc he knows the break up is on its way (wich instigates another break up), the way he threatens me about not breaking up with him - or else he will start another relationship with someone else, the way he throws up to me that he always buys me things and I don't buy him as many things, the way he says I always break up with him before holidays or his birthday to intentionally mess his plans up or to intentionally hurt him, he won't ever apologize or see his part in it unless we have to go through a knock down dragout and after me calmly explaining to him how it hurt me and what could have been done differently, then he may say he's sorry and to please not break up with him bc I'm "teaching" him things,

Couple of questions:

Is it wrong of me to not go out with him saturday night? Here are the circumstances preceding my decision not to join him - I was planning to go with him even though I was scared since it had been awhile for me to be at a club (upper class lounge - not like there was dancing and loud music or anything). I wanted to show him that I could and would do it for him since he always told me how crappy and selfish I was to not want to go out with him. Circumstances: we had argued the night before because I was supposed to go to his house and spend the night. It took me longer than I thought to get my things together etc. So I called him and left him a message telling him to not wait up for me and to go onto sleep, that I was running behind and would be there as soon as possible. He called me right back and was short with me. I could barely hear him because the tv was turned up so loud (he didn't bother turning it down - he was punishing me) I told him what I had just left on his voice mail and he was sarcastic and got off the phone without saying bye. I was calling to let him know all of this to be nice. I had nothing buy good intentions. It made me mad that he was rude not to turn the tv down and for being short with me just because I wasn't there at the time he had hoped me to be there (there was no specific time set) I always go over to his house to spend the night). So I called him back and asked "are you going to be nice to me when I get there?" I didn't yell and was simply asking bc if he was perturbed enough to be rude on the phone then I surely didn't want to go over there late at night and sacrafice more of my time only for him to be rude when I got there. As soon as I asked him if he was going to be nice when I got there, he went off! He started yelling at me about how I was making excuses like I always do and like I was trying to weasle my way out of coming over and that if I was coming over fine, if not fine but that I was so stupid to always play stupid games and look for excuses not to come over. (all of this was yelling and many curse words were mixed in there) first of all, I never make excuses because of anything with him, I may try to be extra nice when I do tell him I can't do something exactly as he wanted me to etc but that's only bc he's gotten so furious with me for telling him any other way. In the midst of him yelling at me, I started crying and hung up on him. The next morning he called and we didn't really talk too much about the situation. (I'm exhausted from arguing with him and trying to point out the way he makes me feel). We hardly spoke that day (saturday) and when I went to his house that evening dressed and ready to go, he acted standoffish and weird - first of all, he had changed our plans and didn't tell me, second of all, when I asked any questions about our plans, he acted like it shouldn't matter to me, and thirdly - it was almost like he was just waiting on my to "bail" on him and once again ruin his plans. He knows how insecure I am about going out and he's surely not ever made me secure about him so the combination was too much to handle - especially when he offered no reassurance of what we were going to do and that it would be fun and that he would be with me and for me not to worry etc...so we ended up getting in an argument and I left. The next day I apologized for making him feel like I did it to be mean and selfish. I tried to explain the reasons for my insecurities again but he didn't listen nor care. That night I went to his house and I told him in person that I was sorry for hurting him by not joining him the night before. When I apologized, he told me not to talk about it. I had a weird feeling in my gut about what he did saturday when he went out (without me) we got along great that night and
Then I went home. I called him before I went to bed and out of the blue he asked me "would you ever have sex with someone else, ya know - like a one night stand?" I said no of course which is the honest truth - I'm not into the social scene nor am I into dating more than one person at a time and definitely not into one night stands and/or sex with someone else. That immediately made my gut hurt and heart race more but I didn't say anything because I know if I did, he would only tell me how I shouldn't be asking him and beat around the bush and make it my fault somehow. Sometimes I can't bear to actually "hear" the words he tells me over the phone when we are talking about something very serious such as this because he gets defensive and mad instead of simply answering the question and putting my mind at ease. I sent him a text message after we hung up and said "I have had a weird feeling in my gut all night tonight and then we you asked me that quesiton out of the blue, it really made me worry. Did you do something that would have been out of line last night?" he replied back with "I did not have sex with anyone last night." I replied back with "there are other things you could've done that don't have to include "sex". Did you do anything?" then he called and went off about how I had no right to even ask him about saturday night - that I should have thought about that when I walked out on him saturday, that if I was worried about that, I shouldve called saturday after I left etc." ...sooo that led to me breaking up with him again. He talked to me like a dog and wouldn't even let me ask a question about saturday night bc I didn't join him in going out as planned. Is that right? Not only would he not let me talk about it, he instead told me how selfish I was and how I always ruin his plans and that we have never gone out and that this is just one thing he asked of me and I couldn't even do that. That he lost respect for me for not doing what I said I was going to do. That I was a flakey bit*h who had mind issues and who liked to play games (I hate to play games by the way) I told him I didn't want a relationship where I had no right to talk about something that happened just because I didn't follow thru on my part. After I told him that, he started calling all of my phones over and over and over again - like he always does. Now, if ever in the past, after I had broken it off with him, he would've said something nicely and adult like when I answered, I would've answered this time. But he's never, never said anything but how much of a piece of crap I am when he calls me after I break up with him. Should I answer the phone if I know he's only going to yell at me? Then, I get an email from him saying his grandfather passed away so I reply and tell him that I am sorry. I also left him a voice mail saying I was sorry. He called me back and launched into everything I said he would say...etc. This is only one instance. There have been instances where he's put his hands on my throat, pushed me, threatened me etc. Of course I know that that is screwed up and that that alone justifies me not being with him. But in this question, will you please respond regarding the other stuff I'm talking about? Not the physical, but the emotional,psychological part please? I need to know if I'm crazy. Thank you so much.

wow I read all of that and all I have to say is you guys don't belong together. I started answering some of your questions as I read, by the time I was done I deleted it all to say you guys don't belong together. I don't even see how someone could put up with that. If your not crazy now you will be. are you afraid you can't find another man because you don't like the social scene. F*ck that, im sure a girl like you would be fine. Good luck and I hope you don't put up with that sh*t to much longer. People don't deserve to be treated like that. Sorry I don't have more advice but I don't answer questions that p*ss me off, I really don't know what to tell ya. He's seems like a f*ckin scumbag that landed a good girl, seems to happen all the time. You really need to stand up for yourself before something bad happens. Your not in the wrong he is, the quicker you realise that the better off you are. If you need to talk some more drop me a funmail. I won't critisize you