How to tell relatives they can't stay at your home anymore?

My brother (btw: who is isn’t my mother son but we have the same father) stays out the state, and for the past few months when he has visits he stays at my house. (also we haven’t seen each other in years and just reconnected with each other last year) but when he goes out he comes back in late wee hours of the morning that means if noone hears him he knocking at windows, and calling our phones so one of us can get up answer the door. (last time he almost lost a set of keys) any time before we reconnected he stayed at his aunt house which he says is to noisy and can’t get any rest and his aunt tells him to make it back by a certain time. My moms always gave us rules about what time to be home for, and if we knew we were going to be gone later than expect we take a key. Plus we don’t like a lot of activity here with people in and out. He also says he stays here to spend time with us which isn’t true because I’m in school half the say and he stays’ gone all day and night. (some times he pops at the house unexpected, and calls say he like a hour or so away and still comes extra late.) I want to know how to tell him he can’t stat here anymore after the mardi gras holiday here?