Introduction
A/B testing is used to compare different variants of a website or feature in order to evaluate their impact on user behavior and business outcomes.
In the context of recommendation systems, A/B testing is commonly used to compare different recommendation strategies, placements, or providers.
A/B test tracking and analysis are performed using Mapp Intelligence.
Mapp Fashion delivers recommendation content but is not responsible for tracking.
Typical use cases include:
Comparing Mapp Fashion with a competitor solution
Testing different recommendation strategies (for example, Similar Items vs. Outfits)
Optimizing recommendation placements and widgets
Measuring impact on conversion rate and revenue
Core Concept
A/B testing consists of two parts:
Variant assignment
Each user is assigned to a specific variant (for example, Variant A or Variant B).
Tracking and analysis
User interactions and conversions are tracked and analyzed per variant.
Tracking is always handled by Mapp Intelligence. This includes:
Page views
Product interactions
Orders and revenue
(Optional) recommendation interactions
When testing recommendation performance, Mapp Fashion provides the recommendation content, while Mapp Intelligence ensures that all interactions and outcomes are measured and attributed correctly.
Tracking Approaches
Mapp Intelligence supports three approaches for tracking A/B tests.
They differ in implementation effort, level of detail, and attribution capabilities.
1. Integrations with A/B Testing Tools
This approach is based on passing test and variant information from the A/B testing tool to Mapp Intelligence.
While Mapp Intelligence provides direct integrations for selected tools, the same principle can be applied to any A/B testing solution that exposes variant information.
In general, the A/B testing tool assigns a variant to the user and makes this information available (for example via JavaScript or a data layer). This value can then be included in Mapp Intelligence tracking.
How it works
The integration reads test and variant information from the A/B testing tool
Variant data is automatically sent to Mapp Intelligence
Data is available in dedicated A/B test dimensions
Advantages
Low implementation effort
Standardized data structure
Automatically linked to all interactions and conversions
Limitations
Only available for supported tools
Not applicable to custom or proprietary testing setups
When to use
A supported A/B testing tool is already in use
Fast and low-risk implementation is required
For setup instructions, see:
2. Recommendation Tracking (Mapp Fashion context – recommended)
This approach uses Mapp Intelligence recommendation tracking to measure interactions with recommendation content.
Variant information is included in the tracking context (for example, as part of a parameter value).
How it works
Recommendation interactions (such as clicks) are tracked via Mapp Intelligence
Variant information is included in the tracking data
Product interactions and orders are linked to prior recommendation interactions
Advantages
High level of detail
Direct attribution from recommendation interaction to purchase
Suitable for comparing recommendation strategies or providers
Limitations
Requires implementation effort
Tracking may be limited when content is rendered inside iframes
When to use
Recommendation performance needs to be analyzed in detail
Different recommendation engines or strategies are compared
Item-level attribution is required
For implementation details, see:
3. Custom Parameters
This approach works in all technical environments and is often used as a simple and reliable way to run A/B tests. It is based on passing test and variant information using custom parameters.
Depending on the implementation, different parameter types can be used, such as page parameters, event parameters, or e-commerce parameters. However, this approach provides less detailed analysis compared to recommendation-based tracking.
How it works
The website assigns the active test and variant
Values are sent as parameters (for example, ABTest and ABTestVariant)
Analysis is performed using filters and segments in Mapp Intelligence
Advantages
Works in all technical environments
Simple and robust implementation
Limitations
Low level of detail
No direct attribution to recommendation interactions
Requires consistent parameter handling
When to use
Simple A/B tests where ease of implementation is a priority
Other approaches are not feasible
A high-level comparison of variants is sufficient
A third-party solution cannot be tracked directly
For implementation guidance, see:
Comparison Overview
Approach | Granularity | Purchase Attribution | iframe Compatibility | Implementation Effort |
|---|---|---|---|---|
A/B Testing Tool Integrations | High | Limited | Limited | Low |
Recommendation Tracking | Very high | Yes | Limited | Medium |
Custom Parameters | Low | No | Full | Low |
Note
iframe compatibility becomes relevant when content is rendered inside iframes, as interaction data may not be accessible. In such cases, detailed tracking may be limited.
Choosing the Right Approach
Use integrations with A/B testing tools when available
Use recommendation tracking for detailed recommendation performance analysis
Use page parameters when no other option is feasible