What's the best thing to do now?

My ex-boyfriend and I have dated for 1year and some months, in a on/off relationship. Since the beginning is was very hard to ever to happy, we had a lot problems with trust. I have forgave him plenty of time after being unfaithful or other reasons. We have always accepted each other. Before I left on vacations for 2 weeks, he had talk to my mom clearly that this time it was serious and what-not. AND when I left I told him when we had broken up I had been talking to a guy ( I was single of course) to forget him and move on. This guy was his old-middle school friend however, I did this to get back at all the times he cheated on me, revenge. 4 weeks now that im back he acts all strangely, once I had told him about me talking to the guy I had told him if he was okay or wanted to stop talking to me yet he said no he forgave me. Till now he tells me he is bothered by that, that he doesnt love me, that is disgusting, and he doesnt look at me the same. Yet he always tries to grab my attention and wears the braclet that symbolized our relationship. Mind you me and the other guy never did anything out of place or anything. we just talked and popped kiss once. Should I let it go like he is telling me? We have always separate for space and come back but never has he told me this I usually am the one to rub what he has done on his face. I know he is hurt but its the first time I talk to a dude just to bother him.