PART 3: Saas Tools Vs. CMS Tools in Website Optimization

Implementation strategy to leverage CMS and SaaS website optimization tools

In a previous post we talked about what to consider when selecting a tool to help you continuously optimize your website. At the end, we established that, as a third option, there is nothing stopping you from using SaaS and CMS website optimization tools together to fulfill your marketing needs.

Don't worry if you've already invested in a CMS with digital marketing capabilities but want to take advantage of the exciting features that SaaS tools have to offer. There is a way for them to work together and it can save you money too.

As a marketer or website owner wanting to optimize your website, you will experiment using testing and personalization.

Testing

Website optimization testing (A/B or multivariate testing) is a process where you will:

1. Identify goals: A goal is the metric that you will use to determine whether a test variation of some aspect of your site is more successful than the original version.
2. Generate hypothesis: you test ideas regarding changes that can be made so the user experience might improve.
3. Create variations: make the desired changes to an element of your website experience. 
4. Run experiment: kick off your experiment and wait for visitors to participate. Users are randomly presented with a test version or the original. Their interaction with each version is measured and compared to determine how user interactions on each version compare. 
5. Analyze results: data from the experiment will show you the difference in performance between the versions of your page and whether this difference is statistically significant.

If your experiment generates a negative result or no result, you won't modify the original page version. Instead you will use the experiment as a learning experience and generate new hypothesis that you can test. Constructing hypotheses and iterating will generate multiple content variations. Over time you will be able to determine which variation is the winner. Then you can stop the experiment and present the winning version to all your visitors. As result you have a more successful content variation.

Personalization / Audience Segmentation

Implementing personalization requires adding an extra step to the process previously shown. Before creating content variations you must segment your audience so that only users matching the specified criteria are presented with the tailored content. Like with testing, it will take time to get results and most likely you will conduct a few experiments with statistically insignificant results. Eventually you will be able to identify profitable audiences, and these will be your primary focus for content experiments.

When should you use one tool type over another?

At first your website optimization experiments will be:

• volatile or temporary if you are: 
  • testing multiple content variations that you often modify expecting to get increased conversions.
  • experimenting to determine profitable audience segments.

Over time it will become:

• stable or permanent when you have: 
  • winning content variations based on the analytics collected from your experiments.
  • properly identified target audiences for your business.

If your website optimization is in the volatile or temporary pairing you should use SaaS. If it is stable or permanent pairing you should use CMS tools.

Your website optimization strategy will transition from the left to the right. As you determine which outcomes from the larger set of volatile or temporary are significant, you will want to make the content variations permanent in your CMS. As your audience segments / personas from the SaaS tool increase your revenue, you can make them permanent inside the CMS, connecting them to your actual user profiles.

For example, let's say you run an e-commerce company and want to increase the number of completed checkouts (we can identify this as a goal). To accomplish this, you decide to A/B test the checkout funnel components (the variations test your hypothesis). You use a SaaS tool to create a new version of the checkout funnel (this is the variation), split the traffic equally between the original page and this new variation, set a duration and start your test. Once your experiment is complete, you analyze the results. Your variation wins in terms of successful goal completions. you want to apply what you learned from the experiment to continue iterating to improve your results. After multiple experiments, you have a version that beats others and you decide to make this version permanent. At that point, your development team can modify the page layout in the CMS accordingly.

Another objective for your company could be to increase sales (goal). So you decide to personalize your visitors experience by designing audience specific homepage promotions (based off of your hypothesis). You use the SaaS tool to target your visitors based on their geographic locations (we can call these segments "audiences.") Then create a product offer (variation) for each audience and start your experiment. From the collected data you determine if the product you are promoting sells well for that geographic area. Then you apply what you have learned and continue iterating so that you can continually try to increase your sales. After some time you determine that your hypothesis is true: showcasing products per geographic area increases sales. Now you are able to identify product preferences per location so you decide to make your audience segments and experiences permanent. Then you ask your development team to provide you with the required criteria to segment your audience and allow the homepage modules to be personalized within the CMS.

Applying the previously described implementation approach, the website optimization process will look as follows: 

Benefits

The benefits of following this implementation strategy are:

1. You are rapidly getting your marketing ideas and experiments to market without a reliance on your IT staff.
2. You save money by:
   1. Not using your developers' time to implement features for the CMS in order to test a hypothesis that could be wrong.
   2. Not regularly paying the SaaS vendor for experiments and audience segmentation that have been proven to work and that you want long-term on your website.

Pitfalls

This strategy is not without potential issues, however. You need to have mature marketing and IT departments individually AND cross-functionally to make this approach work for your organization on a large scale over time. If everyone in the organization is starting with the following two goals then this is possible:

1. Meeting the business need for speedy and constant improvement.
2. Using company resources (people and money) as effectively as possible over the long term.

The considerations that need to be addressed and answered are:

1. When do you define an experience as fitting the volatile or temporary pairing and when is the right time to consider it permanent or stable?
2. Are you okay with managing content on multiple platforms (SaaS and CMS) for a period of time?
3. Does your company privacy policy allow you to store content and the resulting analytics using a third party platform?
4. Is it possible to implement your hypothesis using the features provided by a SaaS tool?
5. Can you afford the cost of the SaaS tool subscription in addition to the CMS?