Are looks what matter?

well I started dating this guy like ... hmmm, I don't know a month ago? give or take a few days. anyway, I'm not really attracted to him. let me explain:
I met him 5 months ago and @ first we hated eachother (or so I thought). he was the trouble making, emo, not my type and I knewi could do better, not that I was thinking about dating him then.
we started writing notes to eachother, we both thought it was kinda immature but it was fun and he is so much different on paper. well it turns out that we h ave A TON in common, same favorite band and type of music, same fav books/movies/video games/ poems... you name it. and most importantly we both had a boyfriend/girlfriend that was killed last january. mine commited suicide and his was shot and we are both just getting over it. when I first met him she was all he thought about, oh and did I forget to mention I look just like her? well I do.
well long story short he asked me out many times I even though I wanted to say yes I never did. I guess it was because I knew my friends and everyone else would start talking about us and I was afraid of being heart broken again.
one day I decided to say yes to him and now we are together. I really like him and I know he loves me. even though he's no tthe cutest guy I've dated he means the most to me.
but everytime I hear someone talking about how creepy he is or how he's ugly (which in all honesty he's not ugly!! he's got black hair and pale skin and green and golden eyes.) I start to feel all... I don't know horrible. like I don't want them to find out im with him. and then I realize that I NEED him!! he is more important to me then almost anyone else. I never thought I was so shallow but I guess I was wrong.
how can I fix this? advice please?

Looks is something that can be changed, because so many people worry about looks, 'oh do I look fat in this size 2 dress?' what people miss is the true person inside, the person who is inside, not some space whom you can talk about. Looks never last, but the soul dose, the personality dose, the phycie dose, the memories will stay, but we're transfroming from the second we're a single cell to the day we're some fossil. Nothing is more important then who he is, you shouldn't care if he had no face. There is no 'perfect' guy. But if you love him, say 'Screw/F**k you' to the people who belive he is ugly. (if this make no sence, it's 12am so I'm a bit tired)

yeah that makes sense, thanks!