I switched to Tamoxifen and have done fine. I would definitely call your Onc's office and arrange for a telephone conference. I feel like I have to be on something (I've been on Tamo 5 years, and now told another 5 years). My ER number was 100.
Having said that, quality of life does have to step in sometimes.
HUGS and prayers. Hope you find a resolution soon.