Introduction
A/B testing is used to compare different variants of a website or feature in order to evaluate their impact on user behavior and business outcomes.
In the context of recommendation systems, A/B testing is commonly used to compare different recommendation strategies, placements, or providers.
A/B test tracking and analysis are performed using Mapp Intelligence.
Mapp Fashion delivers recommendation content but is not responsible for tracking.
Typical use cases include:
Comparing Mapp Fashion with a competitor solution
Testing different recommendation strategies (for example, Similar Items vs. Outfits)
Optimizing recommendation placements and widgets
Measuring impact on conversion rate and revenue
Core Concept
A/B testing consists of two parts:
Variant assignment
Each user is assigned to a specific variant (for example, Variant A or Variant B).
Tracking and analysis
User interactions and conversions are tracked and analyzed per variant.
Tracking is always handled by Mapp Intelligence. This includes:
Page views
Product interactions
Orders and revenue
(Optional) recommendation interactions
When testing recommendation performance, Mapp Fashion provides the recommendation content, while Mapp Intelligence ensures that all interactions and outcomes are measured and attributed correctly.
Comparison Overview
Approach | Granularity | Purchase Attribution | iframe Compatibility | Implementation Effort |
|---|---|---|---|---|
A/B Testing Tool Integrations | High | Limited | Limited | Low |
Recommendation Tracking | Very high | Yes | Limited | Medium |
Page Parameters | Low | No | Full | Low |
Note
iframe compatibility becomes relevant when third-party solutions (for example, recommendation providers) run in isolated environments and do not expose interaction data. In such cases, detailed tracking may not be possible.
Tracking Approaches
Mapp Intelligence supports three approaches for tracking A/B tests.
They differ in implementation effort, level of detail, and attribution capabilities.
1. Integrations with A/B Testing Tools
This approach uses integrations with supported A/B testing tools such as Optimizely, AB Tasty, or Kameleoon.
How it works
The integration reads test and variant information from the A/B testing tool
Variant data is automatically sent to Mapp Intelligence
Data is available in dedicated A/B test dimensions
Advantages
Low implementation effort
Standardized data structure
Automatically linked to all interactions and conversions
Limitations
Only available for supported tools
Not applicable to custom or proprietary testing setups
When to use
A supported A/B testing tool is already in use
Fast and low-risk implementation is required
For setup instructions, see:
2. Recommendation Tracking (Mapp Fashion context – recommended)
This approach uses Mapp Intelligence recommendation tracking to measure interactions with recommendation content.
Variant information is included in the tracking context (for example, as part of a parameter value).
How it works
Recommendation interactions (such as clicks) are tracked via Mapp Intelligence
Variant information is included in the tracking data
Product interactions and orders are linked to prior recommendation interactions
Advantages
High level of detail
Direct attribution from recommendation interaction to purchase
Suitable for comparing recommendation strategies or providers
Limitations
Requires implementation effort
Limited if a third-party solution runs in a closed iframe or does not expose interaction data
When to use
Recommendation performance needs to be analyzed in detail
Different recommendation engines or strategies are compared
Item-level attribution is required
For implementation details, see:
3. Page Parameters
This approach tracks A/B test information as parameters with each page view.
How it works
The website assigns the active test and variant
Values are sent as parameters (for example, ABTest and ABTestVariant)
Analysis is performed using filters and segments in Mapp Intelligence
Advantages
Works in all technical environments
Simple and robust implementation
Limitations
Low level of detail
No direct attribution to recommendation interactions
Requires consistent parameter handling
When to use
Other approaches are not feasible
A high-level comparison of variants is sufficient
A third-party solution cannot be tracked directly
For implementation guidance, see:
Tracking guidelines for page parameters
Choosing the Right Approach
Use integrations with A/B testing tools when available
Use recommendation tracking for detailed recommendation performance analysis
Use page parameters when no other option is feasible