Ecommerce Conversion Rate Optimization: The Program, Not the Tips

Ecommerce conversion rate optimization is easy to run as a list of tips: add reviews, simplify checkout, make the button bigger. Each of those tips assumes you can prove whether it worked, and at 40,000 sessions a month and a 1.4% conversion rate, proving a 5% lift takes nearly two years of traffic. We run CRO programs for Shopify stores, and this is how we’d structure one for a client. Measurement comes first, then benchmarks read with their denominators attached, then research and fixes everywhere, and controlled tests only where the arithmetic allows.

Nearly every data source in this field sells analytics, testing software or services. The sample-size and expected-value arithmetic is our own, run off published formulas.

Conversion rate optimization is the work of raising the share of visits that end in a purchase without buying more traffic to do it. Shopify defines conversion rate (opens in new tab) as the percentage of online store visits (sessions) that resulted in a sale. That definition counts sessions, not people, and says nothing about what each sale was worth.

Revenue per session is roughly average order value multiplied by conversion rate, so a change that lifts one while suppressing the other can leave revenue flat. We judge CRO work on revenue per session. Our guide to raising average order value on Shopify without wrecking your margin works through that arithmetic.

A tip says ship the change and watch the dashboard. A program first asks whether your traffic can detect the change at all, then routes it: straight to production when the evidence already supports it, into a controlled test when the math allows, and back to research when neither holds. The route also sets how you’ll report the result: a controlled comparison for a test, a before-and-after read against your own baseline for a fix.

Every conversion rate is a numerator over a denominator, and configuration moves both. In GA4, a session ends or times out after 30 minutes of user inactivity (opens in new tab) by default, and session and user metrics are calculated through an estimation. Change the timeout and the denominator changes, which is one reason GA4 and Shopify won’t report the same session count for the same store.

The numerator moved as well. Google now calls events that measure actions that are important to the success of your business “key events.” (opens in new tab) A GA4 “conversion” now means an action used for Google Ads measurement and bidding, so a GA4 conversion rate can include key events that aren’t purchases. Check which events feed the rate before you compare it to anything.

Then confirm the purchase event fires at all. If your purchase tracking lived in Shopify’s Additional Scripts field, check it against what broke in Shopify conversion tracking on August 26 before you trust any rate on your dashboard. A test read off a broken purchase event is wrong in both arms.

Each published average ecommerce conversion rate below is a vendor’s number, measured on the vendor’s customers with the vendor’s denominator. Littledata, which sells Shopify analytics, benchmarked 2,800 Shopify sites in 2023 (opens in new tab) and put the average at 1.4%, with the top 20% above 3.2% and the top 10% above 4.7%. Littledata publishes the population and the year. It doesn’t publish how it defines a session or a conversion, the date range within 2023, or when the figures were last refreshed.

Dynamic Yield, which sells personalization and experimentation software, puts the average ecommerce conversion rate globally at 2.72% (opens in new tab). Its formula divides completed actions by the number of users that visited the site during the same period, which makes the denominator users rather than sessions. The data is aggregated across Dynamic Yield’s customer base, which includes 200M monthly unique users collected over 300M total sessions, over the past twelve months, with a month-by-month breakdown that ran through July 2026.

Contentsquare, which sells experience analytics, draws on 6,500+ websites across 9 industries (opens in new tab) and 99 billion sessions analyzed across web and mobile, comparing Q4 2024 with Q4 2025, for its Digital Experience Benchmark 2026. Its conversion finding is “Conversion rates dropped -5.1% this year.” Contentsquare doesn’t say whether a conversion means a purchase or any goal, and the sample spans nine industries rather than ecommerce alone. Contentsquare itself cautions that “Benchmarks reveal trends, not standards.”

IRP Commerce, an ecommerce platform vendor, reports an all-market conversion rate of 2.26% (opens in new tab) for July 2026 against 1.94% for July 2025. The scope is B2C ecommerce in Great Britain, Northern Ireland and Ireland, calculated from trading data on IRP’s own platform, with each order credited to the last recognized traffic source. It’s a UK and Irish platform-customer figure, not a reference point for a US DTC store.

If you need a range, the defensible one runs from Littledata’s 1.4% on Shopify sites in 2023 to Dynamic Yield’s 2.72% per user across its own customers over a trailing twelve months, with those qualifiers attached. Averaging the four mixes populations and windows into a number no store measured.

The vendors disagree on which device converts better. Littledata puts Shopify mobile at 1.2% against desktop at 1.9% (opens in new tab). Contentsquare reports desktop conversion rate was 74% higher than mobile (opens in new tab), with mobile making up 69.9% of all traffic. Dynamic Yield (opens in new tab) shows the reverse, with mobile at 2.88% against desktop at 2.37%, which by our arithmetic puts mobile about 21.5% ahead.

Each vendor has a different population, denominator and window, so all three can be accurate about their own data and still say nothing about whether your mobile experience underperforms. Compare your own mobile and desktop rates over time, split by new and returning visitors, before you conclude anything about mobile conversion.

A user-denominated rate runs higher than a session-denominated rate for the same store, because one user can have several sessions, and Shopify counts sessions while Dynamic Yield counts users. Windows differ too: a Q4-to-Q4 comparison, a trailing twelve months and a single calendar year from 2023 measure different seasons.

Category moves the rate as well. In Dynamic Yield’s own industry spread (opens in new tab), beauty and personal care converts at 5.39% and luxury and jewelry at 0.72%, a gap of more than seven times. The same category doesn’t line up across vendors either: food and beverage sits at 1.5% in Littledata’s Shopify data (opens in new tab) and 4.8% in Dynamic Yield’s.

Contentsquare reports returning visitors converting at 2.9% (opens in new tab) against 1.7% for new visitors, so a store’s rate moves with its new-to-returning and paid-to-organic mix even when nothing on the site changes. Benchmark your ecommerce conversion rate against your own history, by device, by new versus returning, and by channel, and against your own category rather than the market.

On a store without very large traffic, the binding constraint on ecommerce conversion rate optimization is statistical. Ron Kohavi, Alex Deng and Lukas Vermeer give the standard sample-size rule in “A/B Testing Intuition Busters” at KDD ’22 (opens in new tab): n = 16σ²/δ², where n is the number of users in each variant, for 80% power at a 0.05 threshold with equal-sized variants.

Their general guidance is that A/B tests “are useful to detect effects of reasonable magnitudes when you have, at least, thousands of active users, preferably tens of thousands.” Evan Miller, an independent statistician, gives the same rule of thumb in How Not To Run an A/B Test (opens in new tab), with σ² = p(1 – p) for a conversion rate.

Take a hypothetical Shopify store, on assumptions that are ours: 40,000 sessions a month, converting at Littledata’s 1.4% Shopify average (opens in new tab), with an $80 average order value. That’s 560 orders a month, $44,800 in monthly revenue and $537,600 a year. We treat conversion as a session-level Bernoulli metric with an even 50/50 split, the same method behind our analysis of ecommerce personalization on Shopify, so the numbers agree.

At a 1.4% baseline, applying the Kohavi, Deng and Vermeer formula (opens in new tab), detecting a 10% relative lift takes about 112,686 sessions per variant. Detecting 5% takes 450,743 per variant, and 20% takes 28,172. The exact two-proportion formula asks for slightly more, from 0.5% more at a 5% lift to about 8% more at 20%, and the timing table below uses the exact figures. Randomizing by user rather than by session takes more traffic.

Set the test length first and you get the smallest lift the store can detect in that time. Two weeks gives this store 18,462 sessions and a smallest detectable lift of 37.7%, from 1.4% to 1.93%. Four weeks, at 36,923 sessions, detects 26.0% (1.4% to 1.76%), and eight weeks, at 73,846 sessions, detects 18.1% (1.4% to 1.65%).

For comparison, Kohavi, Deng, Longbotham and Xu report in “Seven Rules of Thumb for Web Site Experimenters” (opens in new tab) that at sites like Bing, where thousands of experiments run every year, “most fail, and those that succeed improve key metrics by 0.1% to 1.0%, once diluted to overall impact.” By our arithmetic, detecting a 1% relative lift at a 1.4% baseline needs 22,220,393 total sessions, about 556 months of this store’s traffic.

Evan Miller states the general case in The Low Base Rate Problem (opens in new tab): “If you’re running A/B tests on binary outcomes, and your conversion rate is in the single digits, there’s a good chance you’re wasting your time.” His table puts a change from 1% to 1.1% at 157,697 subjects per branch.

By the same arithmetic, a single page with 5,000 visitors a month at 1.4% produces about 35 orders per arm per month, and a month-long test there can detect only about a 78% relative lift. At 3,000 visitors a month it’s 21 orders per arm and a detectable lift of about 106%.

Intelligems, a Shopify testing app, recommends 300+ orders per test group and 7+ days (opens in new tab), and notes that meeting these minimums doesn’t guarantee statistical significance. At 1.4%, 300 orders per group is 21,429 sessions per arm, or 4.6 weeks of this store’s traffic, and at that size the smallest detectable lift is 24.0%.

Research and fixes run everywhere, at any traffic level. Controlled tests run only on surfaces with enough traffic to detect a plausible lift in a window you’ll wait out, and at this store’s size a sitewide test aimed at a realistic lift doesn’t qualify.

A test that needs 25.1 weeks and starts in September can’t finish before Black Friday. Peak traffic also shifts the new-versus-returning and paid-versus-organic mix the baseline was measured on.

Low traffic also inflates the effects that do reach significance. Kohavi, Deng and Vermeer write in their KDD ’22 paper (opens in new tab) that “The winner’s curse says that the “lucky” experimenter who finds an effect in a low power setting, or through repeated tests, is cursed by finding an inflated effect.” They add that “when power goes below 0.1, the probability of getting the sign wrong (e.g., concluding that the effect is positive when it is in fact negative) approaches 50%.”

Kohavi, Deng and Vermeer tabulate historical success rates: Microsoft 33%, Avinash Kaushik 20%, Bing 15%, Booking.com, Google Ads and Netflix 10%, and Airbnb Search 8%. They convert those base rates into false-positive risks of 5.9%, 11.1%, 15.0%, 22.0% and 26.4% respectively, at a 0.05 threshold counting only significant improvements and 80% power. At a 10% success rate, more than one in five significant results is a false positive even at 80% power.

The authors invoke Twyman’s law: “any figure that looks interesting or different is usually wrong.” Apply it to any vendor case study that reports a 30% lift without a stated sample size.

When a surface does qualify, Evan Miller puts the first rule plainly in How Not To Run an A/B Test (opens in new tab):

The rule guards against peeking. In Miller’s worst-case scenario (opens in new tab), a test of a change that does nothing, on a 50% conversion rate, stopped at the first significant result with a check after every observation up to 150, produced a 26.1% false-positive rate against a nominal 5%. If you plan to look early, the bar gets stricter: for a true 5% error rate, one peek requires a reported significance of 2.9%, five peeks require 1.4%, and ten require 1.0%.

The habit had vendor encouragement: Kohavi, Deng and Vermeer write (opens in new tab) that “Optimizely’s initial A/B system was showing near-real-time results, so their users peeked at the data and chose to stop when it was statistically significant, a procedure recommended by the company at the time.”

Run at least two weeks. NN/g, which doesn’t sell testing software, recommends running for at least 1-2 weeks (opens in new tab) to account for potential fluctuations in user behavior, even with sufficient traffic. Kohavi and colleagues recommend two weeks for a different reason, to look for novelty and primacy effects, while noting in the 2014 paper (opens in new tab) that “In practice, novelty and primacy effects are uncommon.” Neither source makes two weeks sufficient, and the sample size still decides when a test ends.

Check that the split you asked for is the split you got. Researchers from Microsoft, Booking.com and Outreach.io wrote at KDD ’19 that “Sample Ratio Mismatch in most cases completely invalidates experiment results,” and that “approximately 6% of experiments at Microsoft exhibit an SRM” (Fabijan et al. (opens in new tab)). Small imbalances qualify: “a ratio of 50.2/49.8 (821,588 versus 815,482 users) diverges enough from an expected 50/50 ratio that the probability that it happened by chance is less than 1 in 500k.” A chi-square test on visitor counts per arm catches it.

On Shopify, our reasoning is that theme-level, client-side tests can drop a variant’s tracking or redirect traffic unevenly. So we run the SRM check on every test before anyone reads the result.

Each description below is the vendor’s own account of its statistics. Optimizely describes its Stats Engine as sequential, with false discovery rate control (opens in new tab) using a tiered version of the Benjamini-Hochberg procedure. Its project significance setting defaults to 90% (opens in new tab), and Optimizely doesn’t perform additional false discovery rate control across segments (opens in new tab). Its Frequentist (Fixed Horizon) and Bayesian statistics (opens in new tab) are in beta.

VWO describes SmartStats (opens in new tab) as Wingify’s Bayesian-powered statistics engine for A/B testing, and says sequential testing on its stats engine makes built-in adjustments for peeking errors (opens in new tab). Intelligems says it uses a Bayesian statistical model and Monte Carlo simulations to analyze A/B tests, and concedes that the model doesn’t account for intra-week (daily) seasonality (opens in new tab), or other store-specific factors.

Shoplift uses a Bayesian Probability to Win that has to hold at or above a level for multiple days (opens in new tab) before Shoplift assigns a stage. Shoplift describes the 95% bar as “borrowed from large-scale academic studies” and says “Most real ecommerce tests never reach it in a reasonable window.” That’s an honest statement of the traffic problem, from a vendor with a commercial reason to lower the bar.

Shoplift is $99 a month or $888 a year on the Shopify App Store (opens in new tab), “Starting at $74. Based on site-wide monthly unique visitors,” then $399 a month (starting at $299) and $999 a month (starting at $699), with a 14-day free trial. Its visitor counts are based on total monthly website visitors (opens in new tab), not tested visitors.

Intelligems starts at $69 a month (opens in new tab) on the Shopify App Store and reaches $349 a month, where pricing scales based on sitewide order volume. Convert’s Growth plan starts at $299/mo (paid annually) (opens in new tab) and Pro starts at $420/mo (paid annually), priced on monthly tested users, with a $399 monthly-billed option. Every Optimizely plan is individually packaged (opens in new tab), and VWO names its tiers without showing dollar prices (opens in new tab).

Google’s free option is gone: Google Optimize and Optimize 360 haven’t been available since September 30, 2023 (opens in new tab). Tests on the checkout itself also depend on Shopify’s plan gating for checkout extensions, which our average order value guide lays out.

At most traffic levels, research and fixes are the program, and the evidence on where stores lose buyers is deep. Baymard Institute, a UX research firm that sells research rather than testing software, puts average documented cart abandonment at 70.22% (opens in new tab). It isn’t a Baymard measurement: the value is an average calculated based on 50 different studies containing statistics on ecommerce shopping cart abandonment, spanning 2006 to 2025. Baymard also notes that “a large portion of cart abandonments are simply a natural consequence of how users browse ecommerce sites.”

Baymard’s abandonment reasons come from a 2026 survey of 1,083 US adults (opens in new tab) who have shopped online in the past 3 months, reported as the percentage who selected each option, with multiple selections allowed. Among those shoppers, 42% said they were just browsing or not ready to buy, 40% cited extra costs such as shipping, tax and fees, 20% said delivery was too slow, and 19% didn’t trust the site with their card details.

In the same Baymard survey, 18% objected to creating an account (opens in new tab), 17% found checkout too long or complicated, 17% hit errors or crashes, 13% found the return policy unsatisfactory, 12% couldn’t see or calculate the total cost up front, 10% had a card declined, and 9% wanted more payment methods. Read those as the share of surveyed shoppers citing each reason, not as a split of abandonment causes, since the options sum well past 100%.

The product detail page, or PDP, is where a shopper decides whether the product is right before they see a cart. Two of the reasons above can be answered there rather than at payment: extra costs and return policy. Our five PDP tests to run first sequences the product page work, with the metric that settles each test and the documented gap behind it. On a store too small to test them, the same list works as a fix backlog.

Checkout carries the most existing evidence, so it’s where fixes can ship without a test. Baymard benchmarked 344 top-grossing US and EU sites against its 110+ cart and checkout guidelines and found “65% of sites have a performance of “mediocre” or worse” while “only 2% are “good”” (Baymard checkout research (opens in new tab)). Cost disclosure, delivery estimates, guest checkout and payment options map straight onto the survey reasons, and our breakdown of what actually fixes checkout abandonment orders those fixes by the share of shoppers citing each.

Trust signals belong in the same backlog. The 19% who didn’t trust the site with their card, the 13% put off by the return policy and the 12% who couldn’t see a total up front in Baymard’s survey (opens in new tab) all point at recognizable payment options, a visible return policy and a total shown before payment. Some shoppers leave anyway, and recovering them is a messaging job rather than a page change, covered in our guide to the abandoned cart email flow.

In Google’s Vodafone case study (opens in new tab), published March 17, 2021, “a 31% improvement in LCP led to 8% more sales, a 15% improvement in their lead to visit rate, and a 11% improvement in their cart to visit rate.” It was an A/B test in which the two versions were visually and functionally identical, on a telecom landing page rather than a store.

The retail figure comes from Milliseconds Make Millions (opens in new tab), a Google-commissioned study run by 55 and Deloitte across 37 European and American brand sites and more than 30 million sessions at the end of 2019. They report that “A 0.1 second improvement of mobile site speed increases conversion rates by 8.4% for retail sites and 10.1% for travel sites,” and Deloitte adds that for retail, “average order value increased by 9.2% (opens in new tab).”

Deloitte describes the purpose as observing whether there was a true correlation to conversion funnel progress, so the result is correlational, and its metrics aren’t Core Web Vitals.

Our Shopify speed optimization plan covers the field-data method, the Core Web Vitals thresholds and the Shopify-specific fixes. Speed also contaminates tests: a variant that adds an app script can lose because it’s slower, not because its hypothesis was wrong.

Two levers sit beside conversion rate rather than inside it. Our analysis of ecommerce personalization on Shopify runs the same sample-size math against personalization’s controlled evidence, using the same 1.4% baseline. Our average order value guide works through how a free-shipping threshold or a bundle can raise AOV while contribution per order falls, which is why the program keeps revenue per session as its scorecard.

SEO and CRO work on the same pages, and each constrains the other. Traffic is the input to every sample-size calculation above, so organic growth is also testing capacity: at a fixed baseline and target lift, doubling sessions halves the weeks a test needs. The pages search sends people to are also the pages the program works on, so a landing page that ranks for a query it doesn’t answer shows up in your reports as a conversion problem.

Channel mix changes the rate without any page changing. Contentsquare reports returning visitors converting at 2.9% against 1.7% for new visitors, and AI-referred traffic’s conversion rate rising 55% year over year to 1.3% (opens in new tab). A growing share of new organic visitors can pull a store’s conversion rate down while the site itself improves, so segment by channel and by new versus returning before you judge either program.

Testing can cost rankings when it’s set up wrong. Google tells sites running tests (opens in new tab), “Don’t show one set of URLs to Googlebot, and a different set to humans,” and “Cloaking counts whether you do it by server logic or by robots.txt, or any other method.” If a test runs on cookies, keep in mind that Googlebot generally doesn’t support cookies.

For variant URLs, Google recommends a canonical tag pointing to the original rather than a noindex tag, warning that noindex can have unexpected bad effects in that situation. Google also says redirects to a variant should be a 302, not a 301 (opens in new tab), and that JavaScript-based redirects are also fine.

Google also warns that “If we discover a site running an experiment for an unnecessarily long time, we may interpret this as an attempt to deceive search engines and take action accordingly,” and that “This is especially true if you’re serving one content variant to a large percentage of your users.” A 25-week sitewide test on a small store invites that reading, on top of its statistical problems. Small changes carry little risk: Google says they often have little or no impact on that page’s search result snippet or ranking, and our product page testing guide applies the same rules to PDP tests.

A research-driven program generates more ideas than any store can ship, so it needs a ranking method. ICE, PIE and PXL are three named frameworks, and each comes from a party that sells CRO services, training or growth tools.

PIE comes from WiderFunnel, a CRO agency, and its founder Chris Goward. Kohavi, Deng and Vermeer cite Goward’s 2012 book (opens in new tab) You Should Test That as an example of an A/B testing book that gets the statistics wrong. In WiderFunnel’s definitions, PIE scores Potential (how much improvement can be made on the page or pages, prioritizing your worst performers), Importance (how valuable the traffic to the page or pages is) and Ease (how difficult it will be to implement a test on the page or template).

ICE, associated with Sean Ellis and GrowthHackers, scores Impact, Confidence and Ease.

PXL comes from CXL, a CRO training company, in a 2016 post by its founder, Peep Laja. It replaces subjective scores with eight yes-or-no questions: whether the change is above the fold, noticeable in under 5 seconds, adds or removes something, and runs on high-traffic pages, and whether it’s backed by user testing, qualitative feedback, heat maps or eye tracking, or digital analytics. Most criteria score 0 or 1, some are weighted (noticeability scores 2 or 0), and ease is bracketed separately by estimated time. CXL’s case against PIE and ICE is that their inputs are subjective guesses.

All three rank ideas, and none of them asks whether running the test pays. That’s the question we add, and the framing is ours rather than a published method. Expected value equals the annual revenue passing through the tested surface, times the probability the test wins, times the lift if it does, minus the cost of running it. We anchor the win probability on the 8% to 33% success rates in Kohavi, Deng and Vermeer’s table (opens in new tab), and everything else is an assumption.

Break-even revenue through a surface is the test’s cost divided by win probability times lift. Our arithmetic across six combinations:

Run the hypothetical $537,600 store through three hypothetical tests, all inputs assumed. A shipping-cost disclosure touching every order, at a 20% win probability, a 3% lift and $3,000 in cost, has an expected value of $226. A PDP layout test on SKUs carrying 60% of revenue, at 15%, 5% and $6,000, comes out at -$3,581, and homepage hero copy on 10% of revenue, at 10%, 10% and $1,500, comes out at -$962.

Below roughly seven figures of annual revenue through a surface, most tests have near-zero or negative expected value, before counting the fact that the store can’t detect the lift anyway. So the shipping disclosure ships as a fix, because 40% of shoppers in Baymard’s survey (opens in new tab) already cite extra costs, and tests are reserved for surfaces with the revenue and traffic to justify them. PXL’s high-traffic criterion and PIE’s Importance score point the same way.

Put together, this is the order we’d run it in for a Shopify store.

Fixes shipped without a test still get measured. We compare before and after against the store’s own seasonal baseline and state the weakness of that design in the report rather than hiding it.

Ecommerce conversion rate optimization starts with one calculation: how large a lift your traffic can detect, on which surfaces, in how many weeks. For the hypothetical store above, the answer is 26.0% in four weeks, so a sitewide test aimed at a realistic lift can’t settle anything, and the program becomes research, fixes and a short list of tests where the math works. Benchmarks help only with their denominators attached, and the better comparison is your own store last quarter.

You built something worth finding. Now make sure it converts the people who find it. Schedule a free CRO audit and we’ll size what your store can actually test given its traffic, then build the research-and-fix backlog for everything it can’t. A program sized to your traffic, and our conversion rate optimization services take it from there.