How much does the category affect search rankings at launch/long term?

I’m trying to decide if I should list my product under women’s clothing vs beauty. My competitors who rank highly on top keywords are mostly listed under women’s clothing, but the FBA fees for clothing are so much higher.

I want to better understand how the category will affect product launch and also long term if I decide to change the category later.