Unlocking Growth in E-Commerce with Decision Intelligence: A Case Study

I worked for a time at an E-Commerce business, as a data science partner within a User Research team. During my time there, I noticed a heavy use of various key performance indicators (KPIs) but there wasn’t a coherent way to understand how they are all connected - particularly for someone new to the business of E-Commerce. Each product manager typically cared about their particular basket of KPIs. At the same time, there was a drive to come up with a standardized usability metric internally (there is one externally - UMUX-Lite) separate from loyalty metrics like Net Promoter Scores. The key thing about this project was that we wanted to understand how usability impacts business outcomes.

Where we do go from here? As I told students when I taught physics - the first step to solving a problem is to draw a picture. So I created a causal decision diagram of the factors driving revenue as an outcome which an E-Commerce business (say, the stakeholder being the Chief Product Officer) would be controlling as the decision maker.

Assume that the Chief Product Officer of an E-Commerce business made the following decision objective statement to their team:

“Team, our primary goal for the year is to significantly boost our E-Commerce business's revenue by addressing key areas of market reach, customer engagement, and our pricing model. We need to explore and implement data-driven strategies that directly impact these facets, enhancing our sales performance and elevating customer satisfaction. This initiative is crucial for our growth, and I'm counting on each of you to bring forward innovative solutions that will drive our revenue upwards over the coming months.”

The decision team identifies the following outcomes and levers:

• Outcome: Revenue

• Lever: Digital Ad Spend

• Lever: Digital Usability (e.g. through UMUX-Lite)

• Lever: Item Discount

Causal Decision Diagram for an E-Commerce business trying to drive growth.

You’ll notice some things such as loyalty (e.g. as represented by NPS) not being a lever. That is because there are things that drive loyalty, but loyalty is not directly controlled by the decision maker (e.g. the CPO) themself. Usability is a cause of loyalty, but it is not the only cause - changes in price and marketing (e.g. branding) can impact loyalty as well. Loyalty drives returning users, while digital ads can impact new visitors.

There are also multiple conversion rates - this is referred to as the funnel in E-Commerce, where potential customers start from the home page, move to a product page, add an item to the cart, and then place an order through the cart. There can be dropout in each of these steps, and this can be measured using web analytics, so the total order conversion rate can be broken down and quantified. Usability impacts all of these metrics (user friction will increase bounce rates) and not all of these are driven in the same fashion. For example, discounting an item's price would more likely cause a user to add the item to the cart, increasing the add-to-cart (ATC) conversion rate.

Key KPIs typically captured by the business’s web analytics are among the intermediates - such as orders, average order value, returning visitors, etc.

The diagram itself can provide valuable insight - now we have a picture of how all of those messy E-Commerce KPIs are related together and ultimately drive revenue. But the key thing is seeing this in action. And the Billistician is proud to provide a hosted demo of a Decision Simulation of this business problem.

The decision simulation dashboard showcased is an innovative tool that enables e-commerce businesses to visualize the potential impact of strategic decisions on revenue outcomes. The user-friendly interface presents three adjustable levers—Marketing Spend, Usability, and Item Discount percentage—each representing key operational areas where investment and focus can be altered. By adjusting these levers, businesses can simulate various scenarios and immediately see the estimated effect on important metrics such as visitor numbers, conversion rates, orders, and ultimately, revenue.

In this particular simulation, the Marketing Spend is set at $275,000, suggesting a significant investment in advertising and promotion. The Usability lever is adjusted to a high 75%, indicating that the e-commerce site is highly user-friendly and likely to encourage customer engagement and sales. An Item Discount of 10% is applied, which could be seen as an aggressive pricing strategy to attract buyers. The dashboard calculates the outcome of these inputs, predicting nearly 8 million visitors, a conversion rate of 2.5%, and an average order value of $100, which collectively would result in a revenue of just under $20 million. This interactive approach not only aids in decision-making but also provides a dynamic way to predict and plan for future business growth.

To prove that the decision simulation is causal, select “Graph”. Here, you will still see the levers and optimize button, but instead of the list of intermediates and outcomes, you can select a particular decision element in the graph and see how changing one lever changes some elements but not others, for example.

The decision simulation dashboard is more than just a tool—it is a visual story of how e-commerce KPIs interconnect to create a comprehensive business narrative. It brings to light the causal relationships between strategic decisions and their outcomes, allowing leaders like the Chief Product Officer to make informed choices with a clear understanding of their potential impact. This simulation provides an empirical backbone to the often complex and nebulous task of decision-making in e-commerce, demystifying the path to increased revenue through a methodical and data-driven approach.

By integrating usability metrics, digital ad spending, and pricing strategies, this simulation bridges the gap between theoretical KPIs and practical, actionable business strategies. It is a testament to the power of data visualization and simulation in modern business, enabling a proactive rather than reactive approach to growth. This dashboard exemplifies the next generation of business tools, where complex data becomes accessible, actionable, and ultimately, an invaluable asset in driving successful outcomes.