The Power of Business Experimentation

Roberto Croci / January 2019

We live in a data-rich world, where the old challenge of wanting to collect enough data has been replaced by a new challenge for marketers: trying to make sense of too much data. The opportunity for brands? To prioritize and leverage this flood of information to design the kind of relevant and personalized experiences that consumers have come to expect.

Building a Culture of Growth and Optimization

People interact with brands during a series of intent-rich, micro-moments across multiple devices and channels. And each and every moment yields more data to be captured. 

So, how do we become more useful? Today more than ever, organisations need to build a better, more useful experience for their customers and this can only happen systematically by creating a culture that embraces the use of data, testing, and optimization as a means to improve the customer experience every day. We call this environment a culture of growth.

An important part of Innovation is the field of Business Experimentation. Experimentation is the engine that drives Innovation. We’ve seen that leading marketers are the ones who turn data into action to build a better, more useful experience for their customers. In fact, leading marketers in the US  are more than 2X likely than their mainstream counterparts to use site experimentation and strategic experiments.

A culture of growth offers the best path to major gains in providing better customer experiences:

  • More relevant and personalized experiences make people happy
  • As testing proves itself, it tends to generate higher-level investments of support, talent, and resources. 
  • The payoff arrives in the form of more visitors, more sales, happier customers, and a healthier bottom line.

Let’s take a look at the key elements that make up a culture of growth — a culture where everyone is ready to:

  • Test everything, from call-to-action buttons to personalized home pages
  • Run well-planned experiments with clear results that drive action
  • Pay attention to what the numbers say
  • Keep testing and learning daily, monthly, and yearly
  • Fail often, and learn from those fast failures

Experimentation Put Into Action

While many people are focused on success, the failures are just as important and we can learn a lot from them too. So now the question is: "How do we make actually good decisions?" and that's where the world of experiments come in.

Three EMEA wide great examples:

  • We worked on an Unskippable Labs video case study where Mercedes tested 3 configurations to see whether it affected results to mention price in a luxury ad campaign: one campaign didn’t mention price at all, the second included price peripherally within the campaign, and the last had price front and center. Turns out, featuring price was actually the most important factor for millennials considering the purchase of a luxury car.
  • Toyota ALJ in Saudi experimented with an animated series campaign that had fantastic results. The campaign overachieved on all KPIs for reach and engagement. The episodes reached 11.8 million unique users – which represents more than half of the entire online population of KSA – and gained 187 years’ worth of watch time. The series gained vast popularity with an unusually high organic viewership (48%) and exceptional levels of engagement in the form of likes and shares. While some episodes were relatively long, the completion rate for the entire campaign was well above average at 57%.
  • Search is one of the most important acquisition channels in a marketer’s toolkit. But it’s not enough to just optimize search ads. It’s essential to consider the entire customer journey and keep people engaged once they reach your site. For Spotify, reaching the right audience was critical. They discovered that in Germany the most streamed type of content wasn’t music but audiobooks. They wanted to reach users who were searching for “audiobooks” on Google and had come to their site through one of their Search ads. Using the Optimize 360 and AdWords integration, Spotify built a custom landing page to show these users that they have a wide selection of audiobooks, but also that the experience of listening to them is even better with a premium subscription. This site test increased premium subscriptions by 24% (video, article).

Inspired? Here Are 4 Steps To Shift to a Culture of Growth

It starts with small-scale testing on one or two spots, with changes made based on what the data shows. It won’t happen overnight. It may even take one or two years. But start with small steps that can lead to big changes and big impact:

  • Start simple. Begin your testing with insignificant things. You don't want your first tests to change (or break!) your whole platform. Always think of creating frictionless experiences by testing specific actions your customers can do across the funnel from the time they visit your home / landing page, through their navigation and their engagement with the content on your site or mobile app, down to their conversions.
  • Be incremental. Don't redo an entire homepage all at once; test one item at a time. You'll get a much clearer picture of what's actually working. For example - if you are an e-commerce client - think about improving the home / landing page first, then menu & navigation, search, category / product, conversions and form optimization. For each step in the user experience funnel, think incrementally which additional interaction you could improve. Let’s take for example the step when a user interacts with your products, there is a wide array of variants you could think of to make sure you have a value prop at every point in the funnel, including for example category and product pages, allowing users to sort/filter large number of products easily, adding urgency elements, having pricing info above the fold on product pages or secondary CTAs that facilitate x-device, like wishlist, email or call.
  • Be scalable. Make sure those small changes will scale across your site, your company, and your global teams. Remember to A/B test all recommendations and leverage a repository of audiences and signals to inform both your creative and media strategy! A good A/B test starts with having a clear plan defined in terms of objectives, target audience (test vs control group), sample size (if applicable) and which campaigns to leverage. 

It's also easier said than done to find an easy-to-use solution that doesn’t require a lot of time or resources — one that makes it easy to collaborate and share insights. Keep it simple by focusing on the user experience and thinking mobile first.

  • Most companies get more visitors from mobile than desktop, but a recent study showed that conversion rates on mobile are half of the conversion rates on desktop. The result is missed revenue – which risk growing as mobile is becoming the primary device. The good news: you can increase conversions by optimizing your mobile site. This is key in MENA, where the average mobile user in Egypt and Saudi Arabia spends four hours on their phone per day and smartphone and Internet penetration are also at global highs.
  • For example, once you have identified the areas that will help improve your mobile experience, use a framework to prioritise fixes to see where the biggest wins for minimum effort are. You can start by listing the tests you need to carry out on your mobile site, then assign a score for the potential increase of conversions the page will give and assign a score for the value of the page’s traffic for your business and how important it is for ROI. Finally, you can talk to your developers and ask them to assign a score for how easy it will be to design and implement the fix. 
  • We have a new, free online course that will help clients increase mobile conversions –The CRO Course: Win on mobile! It's an educational video course where the first four episodes give stakeholders awareness about why business success is dependent on great mobile sites, and the ten following episodes show the process of speed and conversion rate optimization.

In this new, data-rich world, a growth-minded culture focused on testing and optimization offers a path for making smart and user-friendly choices that will flow straight to your bottom line.

A culture of growth begins with clear goals and small tests, continues with teamwork and adjustments and bigger victories, and carries through down the road for months and years ahead.

The solution you choose will play a major role in shaping and reinforcing that culture of growth and could be the key to your success.




How to ensure your video creative is driving performance