MixMaxer
Help center › Analytics & reports

Profit per offer and the holdout group

Measure money instead of clicks: profit from cost per item, and the true uplift from a holdout group.

5 min read

Profit

Conversion tells you how many people bought; profit tells you whether the offer was worth it. MixMaxer reads “Cost per item” from your product variants in Shopify each time an order arrives and stores revenue minus cost for the lines an offer priced. You see profit per offer in Analytics › Offer performance, per variant in Experiments, and as the KPI “Offer profit”.

  1. In Shopify open a product › variant › Pricing › Cost per item and enter what you pay for it. Do this for the products in your offers.
  2. New orders record profit automatically; earlier orders stay without cost.
  3. In Experiments choose “Profit per visitor” as the decision metric when the variants differ in discount depth.

Holdout group

Comparing “bundle orders” with “other orders” is not a fair experiment: people who buy bundles were going to spend more anyway. A holdout is: Settings › General › Holdout group (Pro) sends a random share of visitors (5–10% is enough) past every offer: no widget, no cart bar, no badges, no automatic discounts. Their browser remembers the group, and their orders are marked. Analytics › Holdout experiment compares the average order value of the two groups over the same days, which cancels seasonality, campaigns and traffic changes.

  • Give it two to four weeks and at least 30 orders in each group before acting on the number.
  • Switch it off (0%) afterwards: every holdout order is revenue without an upsell.
  • Discount codes and Shopify's own discounts still work for holdout visitors; only MixMaxer offers are withheld.

Common problems

Profit is empty for an offer that has orders.

No cost is set on the variants sold, or the orders arrived before you set it. Set Cost per item and wait for new orders.

Holdout AOV is higher than AOV with offers.

With few orders this happens by chance; wait for 30+ per group. If it persists, the offers may be pushing customers to smaller baskets (for example a cheap bundle replacing two full-price items) and it is worth testing a different discount level.

More in Analytics & reports

← All articles · Next: CSV export, the weekly performance email and alerts · Still stuck? Contact support