Crafting Exceptional Experiences A Sitecore XP 10 Guide to Personalization and A/B Testing

Enterprises today have more customer data than ever, yet most struggle to turn it into meaningful, real-time experiences. Delivering a personalized experience isn’t just about knowing a customer’s name but constantly refining what works and what doesn’t. Unfortunately, without incorporating A/B testing, marketing teams rely on assumptions rather than actual data.

Luckily, Sitecore XP 10 provides advanced tools for delivering personalized experiences and testing and refining them for continuous improvement. These features help validate what truly resonates, preventing wasted resources on ineffective strategies. In this blog, we’ll explore the basic steps to get started with personalization and A/B testing in Sitecore so that you can create engaging customer journeys.

Why You Need Personalization and A/B Testing in Sitecore XP 10

Personalization and A/B testing in Sitecore XP 10 can be a total game-changer. Here are a few reasons why:

• A one-size-fits-all experience is no longer enough. Sitecore’s personalization capabilities empower marketers to tailor content based on individual user behaviors, preferences, and demographics, translating to an enriched user experience that resonates with each visitor’s interests.

• A/B testing opens the door to experimentation, allowing marketers to test different content variations, layouts, or calls to action and identify what resonates best with the audience.

• Personalization isn't just about delivering content but crafting targeted marketing campaigns. By understanding your audience on a granular level, you can deliver messages that speak directly to their needs, preferences, and pain points, resulting in more impactful and effective campaigns.

• A/B testing turns assumptions into insights, allowing marketers to make informed decisions backed by real numbers.

• A/B testing isn't just about comparing two or more versions. It allows you to refine your content strategy by identifying the most effective headlines, images, and overall content structure.

• Understanding and adapting to a user's journey is pivotal. Sitecore's personalization capabilities allow you to dynamically adjust content based on where a user is in their journey, providing a relevant experience, irrespective of the stage of interaction.

• A/B testing helps identify high-performing strategies, ensuring that your marketing efforts are creative and efficient by maximizing return on investment (ROI).

Getting Started

Now that we've explored why personalization and A/B testing are indispensable components of a robust marketing strategy in Sitecore XP 10, let's shift our focus to the practical.

Personalization and A/B testing within Sitecore XP 10 should be aligned with strategic objectives, which are then broken down into marketing objectives and digital goals. Sitecore helps to monitor and analyze the impact of marketing initiatives and their relationship with your digital goals.

The first step in Creating, Deploying, and Assigning a goal in Sitecore involves a structured process that seamlessly integrates with the platform's capabilities. Subject to a workflow, goals require deployment before becoming accessible in the Marketing Control Panel for association with content items or campaigns.

Creating and Assigning Goals

To create a goal, follow these steps:

• Navigate to the Marketing Control Panel

• Access the "Goals" section, and employ the "Goal" option in the Insert group to initiate a new goal.

• In the subsequent content tab, details about the goal can be specified, including:

  • Name of the goal

  • Points. Engagement Value Points represent a quantifiable measure assigned to specific interactions or activities on a website that contribute to the overall engagement of visitors. They allow marketers and content authors to assign a numeric value to goals, events, or interactions, providing a measurable way to understand which interactions are more valuable and contribute more significantly to the overall success of their digital initiatives, opening an opportunity to optimize content strategy, and make informed decisions to enhance the overall user experience and achieve specific business objectives. For example, completing a purchase on an e-commerce site might be assigned a higher engagement value than simply viewing a product page.

  • Description of the goal and the associated image for the Experience Profile

  • Select the IsLiveEvent checkbox to submit the goal to the Marketing Automation Engine as soon as it is triggered instead of waiting until the end of the session.

  • Select the IsGoal checkbox to ensure you can associate the goal with a content item. The goal should be available in the Content Editor, in the Goals Attribute section in the Analyze tab.

  • The rule section can be left empty unless you want to select a rule to be evaluated after the page event associated with the goal is triggered.

  • And various other attributes that enhance its contextual relevance. Further checkboxes such as "IsSystem," “Category,” "IsAuthorFeedback," “IsFailure,” and “Show in Events” allow for nuanced customization.

• After entering the goal details, save the configuration and deploy the goal on the Review tab.

• Publish the goal; this ensures its availability for integration into your personalized user experiences.

The next pivotal step in this process is associating goals with specific content items, whether linking goals to landing pages, search fields, registration forms, or media items like PDF files or videos.

• Navigate to the desired page or element in the Experience Editor or Content Editor.

• Access the "Goals" section.

  • In the Experience Editor, it is on the Optimization tab.

    

  • In the Content Editor, it is on the Analyze tab.

    

  • In the Media Library, it is on the Analyze tab for linking goals with media items.

    

  • Select the relevant goal to establish the association.

Creating Component Variations, Setting up Personalization Rules, and Setting up A/B Testing

Now that we've navigated the foundational steps of creating and associating goals with your content items, we'll walk through the step-by-step process of creating component variations, setting up personalized rules, and initiating A/B tests.

In the Experience Editor, creating a new variation of components is a seamless process.

• Navigate to the Versions tab, select the desired language version (remember, run the test in one language version at a time), and choose the component you wish to test. With a simple click on "Test the Component," a modal dialog opens, allowing you to create new variations by clicking the “+” icon.

  

• Name your variation, specify content and design, and initiate a comprehensive content test. To fine-tune the design, enable the variation of component design and select a rendering. Once all variations are incorporated, don't forget to save and publish your changes.

Integrating the "Approve with Test" command into your item workflow is crucial to streamline the testing process. This command will allow you to preview and start a test from the Experience Editor after making changes to your pages using the workflow. You can easily apply this command to your workflow from the Sample Workflow; it must be available to items in the “Awaiting Approval” state.

Moving on, set up personalized rules for your components. First, ensure all your content is ready, including the necessary rendering items. Now, let's dive into creating a personalization rule:

• In the Experience Editor, navigate to the View tab > Enable group > and check the Designing checkbox to activate the design functionality.

• Locate the component you want to personalize, click on it, and on the floating toolbar, hit the Create or edit personalization for this component icon:

  

• This opens the Personalize the Component dialog box. Click the "+" icon to add a personalization rule.

• Select your desired condition from various categories, such as Campaign, Date, Device, Fields, and more. You can customize the condition values by editing the rule field. The order of conditions matters, as they are evaluated sequentially. If the visitor doesn't meet the first condition, Sitecore moves to the next until a match is found.

• Name your condition appropriately, considering clarity. For example, "Only show to local users." When you're done, close the Create rule dialog.

• In the Personalize the Component dialog box, you can control what happens to the content based on the conditions. Navigate the content tree to hide or change the content. Additionally, personalize the layout by enabling the "Personalize the presentation component" setting and selecting a rendering from the Browse option in the Presentation field.

  • The personalization effect tracking feature is in the Personalize the Component dialog box. This feature allows you to assess the impact of personalization rules without executing a full personalization test. When activated, 10% of the component's traffic is designated for effect tracking, meaning that 5% of visitors within the personalization group will be exposed to the default version of the component.

    Within the Personalize the component dialog box, you can enable or disable the Track effect of personalization on this component setting. It's important to deactivate personalization effect tracking if you intend to initiate a personalization test for the component.

• Remember to save and publish your changes, and you will be able to see the personalized content in action on your live website.⠀

Now, we will focus on setting up a Test in the Experience Editor:

• Remember to integrate the "Approve with Test" command into your workflow's "Awaiting Approval" state; saving and submitting an item in the Experience Editor triggers a message. This message informs you that the item is in the "Awaiting Approval" workflow state, presenting options like "Approve with Test," "Approve," or "Reject."

• In cases where you're not using a workflow, you have different options to set up a test:

  • In the Optimization tab, in the Create section, click on “Page test” to trigger the Page test dialog box.

• Once you have clicked either on the “Approve with Test” or in the “Create test” notifications, the Preview and Start Test dialog box appears. It consists of three tabs:

  • Preview Tab:

    • Provides a preview of the test.

    • Highlights the expected effect of changes.

  • Variables Tab:

    • Lists the variables in the test.

    • Displays created experiences.

    • Indicates the percentage of visitors exposed to the test.

    • Presents statistics, including confidence level.

  • Objective Tab:

    • Specifies the test objective.

    • Allows you to choose how to pick a winner.

    • Sets the duration for the test.

    • Clicking "Start a test" initiates the test.

• To make the test live, you need to publish the page containing it. In the Publish item dialog, select "Smart publish" and choose the "Publish related items" option.

You can also create a page test using the Experience Editor and Experience Optimization. In the Experience Editor, click on Page Test in the Optimization tab. In the Experience Optimization dashboard, click on the Create Page test.

• Select the original page to test and the Language variant you want to test.

• Click on the “+” icon to select as many pages you want to test against.

  • You can use an existing page.

  • You can test against the previous version of the page.

  • You can create a new version of the page to test against.

• Click on Review and start to review the test. Here, you can adjust test parameters.

  • Depending on the number of pages to test against and the test parameters, an information bar will show the number of visitors needed to find a test winner.

Reports

Once tests and personalization are set up and running, it is crucial to go over reports. The main reason behind tests and personalization is using the analytics Sitecore gathers and making the best decisions based on data. All reports are located in Experience Optimization and are found in three tabs (Test, Personalization Reports).

The Tests tab shows four sections: Draft tests, Active tests, Historical tests, and Suggested tests. In Active and Historical tests, once a specific test is selected, you can access the Reports tab; in it, you will be able to find the following information:

The first section of the Report is the following:

• The Best page effect represents the relative effectiveness of a particular variation compared to the other variations in the test. It is expressed as a percentage, indicating the extent to which a specific variation outperforms the others in achieving the primary goals. This calculation is done by the difference between the trailing value per visit (The total engagement value calculated by adding all engagement values since the test page was visited) between the original and the best experience in the test. A higher Best Page Effect percentage signifies a more favorable impact; if the original experience has a better effect, then the Best Experience effect and Score will display “--.”

• Confidence reflects the statistical reliability of the observed differences in performance among the variations.

• The Score gained is a measure assigned to the test creator based on the engagement value gained and the importance of the tested page. It will appear in the Performance report in the Experience Optimization.

Then, we have the Engagement tab, where you can find a graph listing the test experiences in order of their performance, as measured by engagement value. The best-performing experience is assigned a value of 100, and the bars in the chart indicate the relative performance of the remaining experiences.

The green bar is for experiences that perform better than the original experience, the red bar is for the experiences that perform worse than the original, and the light gray bar is for the original experience.

If you click on the bars in the Engagement tab graph, a right rail details box will display for each experience:

• Trailing value/visit

• Experience effect: Difference in percent between the trailing value/visit of the selected experience and the original experience

• Preview of the experience

• The Top Converted Goals show the goals triggered by visitors exposed to the test and the number, rate, and change of their conversions.

• Top Clicks to Pages. Shows the name, number of clicks, rate (number of clicks divided by visits, in percentage), and change of the next page clicked in the session after visiting the test page.

• Site usage shows:

  • Bounce rate (the percentage of visitors visiting this page and then leaving the site without viewing other pages) values and changes.

  • Average duration on page value and change

  • The average duration for visit value and change

  • Pageviews per visit (the average number of pages visitors look at in one visit) vary.

In an active test, you also pick a test winner or wait until Sitecore has gathered enough data to make a statistical selection. Once the winner is selected, a summary dialog box will appear. This summary includes a preview of the experience that will be the default experience for all visitors not targeted by personalization. If it is a basic content test or page test, and enough data has been gathered, it will show a Personalization suggestions tab, where suggested personalization rules are listed based on data collected by Sitecore and processed by Sitecore Cortex. You can access these personalization suggestions in the Experience Optimization module.

The Personalization tab shows two sections: Active personalized experiences and Personalization suggestions.

In Active personalized experiences, you will find the following information:

• This table shows an overview of all active personalized experiences. It includes tested and implemented experiences and untested experiences.

  • Each row represents one page with personalization, the number of components with personalization rules applied, and the total number of experiences combining the variations of the different components and personalization rules. The best effect field displays the percentage comparison between the best experience and the default experience.

  • If you click on one of the rows (not in the page link), a right rail details box will appear. In it, you will find an image of the experience, its effect compared to the default, the trailing value/visit, the total number of visits, the names of all personalized components on the page, and the combined rules for all the personalized components in the experience.

Wrapping Up

Personalization goes beyond delivering content. It's about crafting targeted campaigns that resonate with your audience on a detailed level. Understanding their needs, preferences, and pain points transforms campaigns into impactful journeys that leave a lasting impression. Sitecore XP’s personalization and A/B testing features enable enterprises to achieve true personalization.

Oshyn is a certified Sitecore partner with decades of experience supporting clients in all aspects of Sitecore. From implementing and maintaining multiple Sitecore products to building engaging websites and incorporating Sitecore's real-time personalization to create a relevant 1:1 experience.

If you want to maximize your personalization efforts, consider optimizing your website for performance. Our ebook, Optimizing Sitecore Performance, explains how.