I can see why he might feel that way, but I think if he really wanted a family with you, then he'll come round to the idea. If that was my boyfriend though, I don't think I could deal with that, he either wants a family or doesn't, and I wouldn't want to be with him if he didn't want to be with me over that.
Your best bet really is to go to your doctor and talk it over with them, they'll offer you more in depth advice
