Run an A/B Test with Mapp Intelligence

Goal

Compare different variants of a website or feature to evaluate their impact on user behavior and business outcomes.

Prerequisites

• Mapp Intelligence tracking is implemented

• A method for assigning A/B test variants is available (for example, A/B testing tool or custom logic)

• Required tracking extensions are configured (for example, recommendation tracking if recommendation performance should be analyzed)

Procedure

Step 1 – Choose a Tracking Approach

Select the tracking approach that fits your setup and analysis requirements.

For an overview of available options, see: A/B Testing for Recommendation Performance

Step 2 – Assign Variants

Assign each user to a variant (for example, Variant A or Variant B).

Variants can represent different implementations, for example:

• Different recommendation providers

• Different recommendation strategies

• Different page layouts or content variations

Typical approaches include:

• Using an A/B testing tool

• Assigning variants server-side

• Assigning variants client-side

Ensure that each user remains in the same variant during the session.

Step 3 – Set Up Tracking

Ensure that the assigned variant is included in all relevant tracking events.

This includes:

• Page views

• Product interactions

• Orders

Depending on the use case, additional tracking may be required.

For example, when comparing recommendation performance:

• Enable recommendation tracking in Mapp Intelligence

• Track interactions with recommendation elements

For implementation details, see: A/B Testing – Implementation (Mapp vs Competitor)

Step 4 – Validate Tracking

Before analyzing results, verify that tracking works correctly.

Check the following:

• Variant distribution is as expected (for example, equal split)

• Users remain in the same variant during a session

• All relevant events include variant information

Step 5 – Analyze Results

Analyze A/B test results in Mapp Intelligence.

Typical metrics include:

• Conversion Rate

• Revenue

• Engagement metrics (depending on the use case)

For example, when analyzing recommendation performance:

• Click-through rate (CTR)

• Revenue attributed to recommendations

For detailed analysis methods, see: A/B Testing – Tracking & Analysis

Result

• Variants can be compared based on consistent tracking data

• The impact of different implementations can be evaluated using key business metrics

Next Steps

• Optimize underperforming variants

• Run follow-up tests to validate improvements

• Extend A/B testing to additional use cases (for example, content or UI changes)