Losing the buybox against an FBM. Why is that?

have a product with some inventory in FBA, however a competitor that is in FBM (no SFP) has the buybox. The competitor has a lower price but his price is plus shipping, when adding shipping it becomes more expensive than my offer, even so he has the buybox. It is also important to mention that I have a better seller rating, shorter shipping time thanks to FBA and an ODR of 0%. So why aren’t they giving me the buybox? The only thing I could notice using tools to monitor the competition is that apparently the competitor is a sales leader in another product and sells much more than me each month.