I know someone who married a man diagnosed with Asperger's. You could be right about the misinterpretation of facial expressions, body language, tone of voice, etc. I think it's strange that he's not giving you specific examples of things you say/do to suggest you're "going off him," though. Perhaps this has more to do with anxiety/insecurities/low self-esteem than Asperger's? Next time the opportunity arises, you might want to question him a bit further. Ask if you said something, then if you did something, then if you didn't do something, then if a specific thought or memory comes to mind... it may take some digging, but hopefully, the two of you can work through this. Let him know it's important to get to the bottom of this issue, even if he doesn't see the importance of doing so.