Why is he getting upset so easily?

I really dont understand it.. but here it goes.. my guy friend is sorta like my boyfriend we act like were together but were not were just friends.. I called him one night to say good night I spent the whole weekend with him so I really didnt want to talk to him that night.. Next He got mad at me and I told him I didnt care if he was mad at me or not and that upset him he asked if I was talking to his ex girlfriend because I was acting like her.. He said when he doesnt call me all day I call him to find out why and I didnt and that also upset him.. im not sure why hes getting upset with me so easily.. He was with this girl for 5 years.. she told me they broke up because she hated haveing sex with him.. and I know he isnt completely over her so why is he gettin upset? I asked him if he thought I didnt care about him since I was being mean tonight and he said Yes that I wasnt acting like it.. I know he cares about me.. so why is he gettin upset with me so easily?