Is he afraid of commitment?

Me and my boyfriend have been going out for almost 3 months now… It started out really well considering I liked 2 other boys at the time, but thats over now and I made it just about us, I love him so much, but recently I feel he doesnt want to be with me, he swears he does, but I cant help but not trust him on this. When I ask him to come over, he says he has stuff to do at the last minute, everytime this is, and it makes me mad. I do actually expect too much from him, I know his not perfect, but yet I feel he can try harder. Since we been going out I’ve lost my virginity to him and being 1 month later, I dont feel connected to him anymore. I have had people sayin dump him, but im going to regret it if I do, and I dont want to lose him…do you think this is because his afraid of commitment, because its gotten more serious?? or is it because he is confused and dont know what to do, after all I am his first proper girlfriend?? I just want to know whats wrong with him, im gettin mixed signals