Hey there,
I think you should talk to him about it to find out if you guys want the same things. Maybe he is only looking for casual relationships, while you are looking for something more serious. If you talk to him and he does say that he's only looking for something casual, then you need to decide if you are alright with that, and if you're not, then just maintain your friendship with him without having anything more (because there's really not much point in cutting him off if he's just being honest).
Hope I helped, and feel free to talk to me if you want to!
Kyra