I don't think you would have owed the guy anything just because you were giving him a handjob. I suppose he just wanted a little more, to see your breasts. I don't think he should have just assumed that he could see/feel your breasts just because you were giving him a handjob. I don't know whether giving a handjob is less serious than a guy touching a girl. I think it's tit for tat (excuse the pun).
You made it clear to him that you didn't want to do anything more than give him a handjob. So for him to get pissy was a bit pathetic. But heyho, don't let it affect you! I think it's wrong that he just assumed he would be allowed to do that just because you were giving him a handjob. If he was unhappy with you touching him, and him not being allowed to touch you, he could have asked you to stop! But I don't a guy would ask for that.
I hope I helped, take care