Married in love with another

Hello all I am a 24 year old girl, I had a bad relationship with my parent and was a very hatefull teenager. I met a neighbor guy of mine that was 19 years older and married at the time. I babysat for their two kids and got close to him. We never had...

sex but did kiss and touch. I fell in love with him. It ended up he got divoriced from her and we went separate ways. I met my current husband and we had a good relationship. I never forgot my first love though and have thought about him often. Well lately my husband has been depressed and cannot handle any negative things and takes all the emotional and physical aspects for him self all take and no give. It even escelated so that he is consistanly yelling about everything and just complains all the time. It has made me physically sick. Well one day talking with my friends they were like you should call up the first guy you loved. I did and he is remarried but said he has thought about me as well and set up a separate email so he can keep in touch with me. I feel I am still in love with him and just so unsure of the future. I can't stop thinking of him and really want to meet up with him. In the meantime my hubby now realizes I'm not as affectionate as I was but I can't seem to get over how he has become. I'm so confused.