Boyfriend at college - can this work?

My boyfriend and I have been together for five months - and no, that doesn't really sound like much, but we've been living together and are super close. We support each other in the things we do - we're absolutely comfortable around each other and can...

tell each other anything. He's 18 and has just started his freshman year at college (about 3.5 hours away, and he only left a few days ago so we're still in the REALLY hard coping stage.). I'm 17 and I've graduated from H.S. We're still together right now and talking several times a day on the phone and online, but he's not much of a communicator like that. In person he's wonderful and will talk to me and even cry in front of me when he's really, really upset - this is a very quiet, sort of withdrawn guy who doesn't let his emotions out to people generally. When we're together, he tells me that he doesn't want to be with anyone else, he could see himself spending the rest of his life with me, and he doesn't give a sh*t about b*tches and h*es at college. (He's not too fond of girls. BUT he treats me amazingly and says I'm the only girl he could ever care about. Don't worry, he's not sexist ;]) I've been in a serious relationship before and I'm a pretty realistic person - but this time, I can actually see this relationship blossoming into something that encompasses a large part of the rest of our lives. I'm super close with his family, too, and even they see it like that. (These are cynical hippy types haha!) Right now I'm going to be seeing him every two weeks and sometimes only a week gap if I can get a ride down there, so it's not going to be atrocious once we get used to not always being together. But in late November I'm leaving for Virginia for a horse-back riding thing. My boyfriend thinks keeping the relationship going when I leave isn't feasible; in his words, long distance relationships just don't work.
NORMALLY, I agree. But here's the deal. I'll be back in December for two weeks. Then in March I'm going to fly up to surprise him. I'm moving back up around May 15th (when he gets out of school) and he wants us to either find a college together or have me go to his college. I think that could work. He says even when we break up he won't do anythig with other girls and he'll still be waiting for me in the summer when I come back. I'm just so scared that he'll find someone else or just fall out of love with me. I know I need to just be more secure in the whole thing, and it'll obviously be easier once he's come up again in a week and a half and I KNOW he still loves me even though he totally fails at talking on the phone Oh and, he's kind of a partier. Like, drinking and pot. He has a chick friend down there that I totally trust, she's a sophomore and she's been introducing him to some of her friends. (She has a boyfriend and she's the laid-back, I hate drama type of girl - she knows me a little and my boyfriend a lot and I trust her.) Do you think that our expectations for our relationship are realistic? Do I even have reason to be worried? I just want to stop going crazzzyyy over this.
Thanks3

princexx - thanks for your advice. That's what I want to do, but he's stubborn! My best friend reckons my leaving day will come and he won't want to break up anymore. We'll see, I guess.
I left him a three page letter in his dorm that he read his first night there, and he has a bunch of pics of us to put up.

OH AND GUYS - HERE'S SOMETHING I'M ADDING TO THE ORIGINAL POST.
His spring break is March 20 - March 29, that's when I'd surprise him. (Or possibly just make plans so he doesn't lose hope!!) If I can persuade him to spend like $150 on plane tickets or drive (it's like 800 miles), he has February 15 - February 23 off.
If all those dates worked out, here's the schedule:
Late Nov - I leave.
December 20 - Between Jan 3 and Jan 10 I visit.
February 15 - February 23 he visits.
March 20 - March 29 I visit.
May 15 I move back.
NOT BADDD.
Now how the hell do I persuade him!?!