MixMaxer"Test your offers" appears on the listing of almost every bundle app. Most of the time it means "look at the dashboard and change the number". A real A/B test is something else: visitors are split, each visitor sees one variant consistently, checkout charges exactly what that visitor saw, and a statistic tells you when the difference is real.
We build MixMaxer, which has this feature, so we have an interest. We checked every app below on its Shopify App Store listing on 28 October 2026 (and the help center where the listing names the feature), and the list says "could not confirm" where we could not confirm. Send corrections to support@mixmaxer.com and we will update the table.
What counts as an A/B test
Four questions. If the answer to any one is no, you have a dashboard, not a test.
- Split: are visitors randomly assigned to A or B, and does the same visitor always see the same variant?
- Enforcement: does checkout charge the variant's price, or does everyone pay the "main" offer regardless of what they saw?
- Metric: can you read the result as revenue or profit per visitor, not only clicks or add-to-cart?
- Decision: is there a confidence read-out and a stopping rule, or do you eyeball two numbers?
The list
| App | Listing says | Real split per visitor | Checkout charges what was shown | Metric | Decision rule | Notes |
|---|---|---|---|---|---|---|
| MixMaxer | A/B testing | Yes | Yes (Shopify Function per variant) | Conversion, revenue per visitor, profit per visitor | Bayesian probability + confidence level; winner applied automatically, loser paused | Volume, gift, bought-together, mix & match; not fixed bundles yet |
| Kaching Bundles | "Built-in A/B testing" | Yes (per their help center) | Not documented on the listing | Conversion, revenue | Manual read-out | Added 2025 |
| Fast Bundle | "A/B testing" under Analytics | Could not confirm | Could not confirm | Could not confirm | Could not confirm | No help article found describing a split |
| Rebolt | "A/B Testing & Advanced Analytics" (Growth plan) | Could not confirm | Could not confirm | Could not confirm | Could not confirm | Feature named only in the plan list |
| Pumper | No mention | No | — | — | — | Reporting only |
| Hulk Volume Discount Bundles | No mention | No | — | — | — | Reporting only |
| PickyStory (Amplify) | No mention | No | — | — | — | "Recommendation performance" reporting only |
| Shopify Bundles (native) | No mention | No | — | — | — | No analytics beyond orders |
"Reporting only" means the app shows revenue per offer and lets you change the offer; it cannot run two variants at once.
What to look for beyond the yes/no
Consistency per visitor
If the split is per page view rather than per visitor, the same shopper sees 10% on Monday and 15% on Tuesday and buys on the cheaper day. The test then measures nothing. Ask the vendor how the assignment is stored (a cookie or local storage keyed to the visitor is the usual answer).
Price enforcement
This is the hard part technically, and the reason most apps do not offer tests. The variant's price has to be applied in checkout, in Shop Pay and in POS, for that visitor only. In MixMaxer each variant is its own discount configuration inside the Shopify Function; the cart carries the visitor's assignment, and the Function prices accordingly.
Profit, not conversion
A 15% tier will almost always convert better than a 10% tier. That is not the question. The question is whether the extra units cover the extra discount after cost of goods. A test that reads only conversion will pick the deeper discount every time. Make sure the app reads "cost per item" from your Shopify products and reports profit per variant. If your costs are not filled in, fill them in before you test; it takes an afternoon and it changes decisions.
A stopping rule that does not need you
A test ends when someone remembers to look at it. Set the rule up front: minimum orders per variant (30 is a sensible floor), a confidence level (95%), and what happens at the end (apply the winner, pause the loser, send one email). A Bayesian read-out ("probability that B beats A: 97%") is easier for a store owner to act on than a p-value.
What to test first
In our experience the highest-value tests, in order:
- Tier 2 depth: 10% vs 15% on "buy 2". Biggest volume, clearest profit signal.
- Tier 3 as percent vs fixed price: "save 20%" vs "3 for $72". Fixed prices often win for products under $50.
- Gift threshold: free gift at $80 vs $100.
- Bought-together discount: 0% vs 5% on the extras. Sometimes the discount is unnecessary.
Do not test two things at once on the same product, and do not change the offer while the test runs.
FAQ
Can I A/B test a discount on Shopify without an app?
Not properly. Shopify has no built-in visitor split for pricing. Theme-level tests (two sections, a cookie) cannot enforce a different price at checkout.
Which Shopify bundle apps have A/B testing?
As of October 2026: MixMaxer and Kaching Bundles document a real visitor split. Fast Bundle and Rebolt name "A/B testing" on their listings but we could not find documentation of how the split or checkout enforcement works. Pumper, Hulk, PickyStory and Shopify's own Bundles app do not mention it.
How long should an A/B test of an offer run?
Until each variant has at least 30 orders and the confidence level you set is reached, and at least one full week so that weekday and weekend traffic are both included. For most stores that is two to four weeks.
What is a Bayesian A/B test?
A method that reports the probability that one variant beats the other ("B beats A with 96% probability") instead of a p-value. It can be read at any time and is easier to act on.
MixMaxer: one-click A/B tests on any volume, gift, bought-together or mix & match offer, profit per variant, automatic winner. Install from the Shopify App Store · A/B tests help article. Last checked 28 October 2026; corrections welcome.