What causes my young male cat to lose hair?

Hey guys, I have a 3 year old black Siamese male cat, with thin fur. Recently my mom has recognized him losing fur on his left side, both his hind legs and two short lines near his tail(rrump). She says they resemble a shaving mark as when you pet him it feels like it. She’s blaming me because I cuddle him and such and thinks I’m shaving him in various spots when I’m not. I’m not sure if he’s licking that area too often, stress, being chased or just some kind of disease/infection in his body. I do hold him often, he does complain but I stop and let him be, could holding him be stressful for him? P.S: We cannot afford to take him to the vet if that’s the answer. We don’t have the money and hoping it’s not anything bad related for his condition. He’s a healthy cat, skinny, he’s his normal playful and loving self. Just the fur loss is our concern. Thanks! And since he’s black, I can’t seem to find good lighting to take a picture to show more proof of what I’m giving information on. If I do, I’ll try and manage a photo tomorrow o: