Non-monogamous relationships can be a wonderful, beautiful thing, but not if you don't want to be in one. It sounds like there are a lot of incompatibilities and broken trust here. Your boyfriend misled you from the very beginning of the relationship. It makes sense why that would be painful, and it means from the get-go you have a crack in the foundation. You deserved honesty, and you have every right to be upset by what happened.

From the sound of it, you're not okay with the current arrangement (non-monogamy). I will tell you right now that if your boyfriend is truly non-monogamous but you hate that about him, it's not going to work. Sometimes polyam/mono relationships can work, but they have to be mostly free of animosity and also full of intimacy, stability, and trust. It sounds like you struggle with those things already.

Don't make him choose between you. Ultimatums never work, not matter the relationship configuration. Even if he "picked" you, trust me, no one will be happy. You'll resent him for everything that's happened up until now, his ex will probably give him hell, and he'll probably resent you for making him choose.

This is clearly not a good situation and if I were you, I would just leave. Obviously it's up to you, but if you do decide to stay together, ALL of y'all need to work on building trust and fostering honest, open communication, or no one will be happy.