Pregnant with attitudes and don't trust boyfriend he don't like it

My boyfriend and I got back together after seperating for 3 months and not even a month into a new relationship I’m pregnant (about 19 weeks now). Not a bad thing at all we were both excited, but now his ex-wife is getting on my nerves always texting and calling him and not just about their kids. And he goes over her house in the morning while shes at work to take their kids to school. I have a trust issue with him real bad now because of her and because of this I tend to snap at him easily. I cop an attitude with him when he keeps things from me, mostly when it comes to her. I’ll find out somehow from peopletelling or he’ll do something and I’ll ask what’s going on. Then I find out then blow up. I get an attitude when hes late for work, not paying attention to me, smokes now. He told me if I don’t change it were done. I don’t want to lose him but when we get into these arguments I run my mouth and end up saying things I wish I hadn’t. I don’t know what to do. I’m so afraid hes gonna go back to ex-wife again. Other people give me their opinion (people who know us) and lately they question his behavior, but he’ll tell me don’t listen to them. I should believe him when he says he loves only me and wants to be with me. He says if he didn’t he wouldn’t come home to me. But his attitudes towards me make me feel like he doesn’t want to be with me. He told me he hes not in love with me and when I asked him how he would feel if I said that to him he said he would do something to change it, but I’ve been trying. There would be times we would be good for a few days then his ex-wife would say or do something. I don’t know what to do?