Can I A/B test shipping rates or free shipping thresholds? (Essential CRO Split AB Testing app)
Not the actual rates. Shipping rates come from Settings → Shipping and delivery in Shopify, and they're the same for every visitor, whichever version they see.
You can use a Theme test to compare how shipping is presented, for example a "Free shipping over $50" message in theme A and a "Free shipping over $75" message in theme B. Customers will still be charged the shipping rates set in Shopify.