Data Transfer: Consumer

Prev Next

Consumers are used to send the track requests to the Intelligence track server. There are ...

  • HTTP_CLIENT: Saves track requests in memory and sends them to the server using separate parallel threads.

  • FILE: Requests are saved in a separate file and send via a cronjob to the track server in a specified interval.

  • CUSTOM: Create your own consumer to send the tracking requests.

The HTTP_CLIENT consumer is ideal for low and high-traffic websites or when tracking specific pages. It ensures fast response times and efficient processing of tracking requests.

With newly introduced mechanisms, requests can now be collected in the background and sent out in batches at regular intervals. The system also distributes processing evenly across multiple queues, enabling stable performance even as traffic increases.

For websites with consistently high traffic, the FILE consumer remains the recommended option for reliable and scalable handling.

When using the HTTP_CLIENT consumer and sending browser requests via the pixel to the track server directly, it is important to set the max batch size to 1 to avoid processing issues and false analyses in Intelligence.

By default, the attempt timeout is set to 100 milliseconds, with a maximum batch size of 50 requests and a maximum queue size of 1000 requests.

MappIntelligenceConfig config = new MappIntelligenceConfig()
    .setTrackDomain("analytics01.wt-eu02.net")
    .setTrackId("111111111111111")
    .setConsumerType(MappIntelligenceConsumerType.HTTP_CLIENT)
    .setMaxAttempt(1)
    .setAttemptTimeout(100)
    .setMaxBatchSize(1)
	.setMaxQueueSize(1000)
    .setRequestInterval(0)
    .setRequestQueues(2)
    .setForceSSL(true);

Using a FILE transfer for high traffic websites because it helps to reduce load and memory used by your server. Instead of saving the requests in your memory, it writes the requests in a file and saves it there before sending it in batches to the track server. This solution is also more reliable in data collection: If a request attempt failed, it will re-write the failed requests to the file instead of dumping them as done when using HTTP_CLIENT.

Recommendation

We recommend using a cron job to send the requests to the server every few minutes and empty your file to keep it at a manageable size.

MappIntelligenceConfig config = new MappIntelligenceConfig()
    .setTrackDomain("analytics01.wt-eu02.net")
    .setTrackId("111111111111111")
    .setConsumerType(MappIntelligenceConsumerType.FILE)
	.setFilePath("path/to/file")
	.setFilePrefix("MappIntelligenceRequests")
	.setMaxAttempt(1)
    .setAttemptTimeout(100)
    .setMaxBatchSize(50)
	.setMaxFileLines(5000)
	.setMaxFileDuration(30 * 60 * 1000)
	.setMaxFileSize(12 * 1024 * 1024);

Using a CUSTOM consumer to create your own consumer to send the tracking requests. For more information about CUSTOM consumers, see MappIntelligenceConsumer.

class TestCustomConsumer implements MappIntelligenceConsumer {
	public static final String CUSTOM = "CUSTOM";
	public boolean sendBatch(List<String> batchContent) {
		// implement custom consumer logic
		return true;
	}
}
MappIntelligenceConfig config = new MappIntelligenceConfig()
	.setTrackDomain("analytics01.wt-eu02.net")
    .setTrackId("111111111111111")
	.setConsumerType(MappIntelligenceConsumerType.CUSTOM)
	.setConsumer(new TestCustomConsumer());

Please check Configuration to see all configuration options and explanations of the Mapp Intelligence Java library, including default settings.