Prepare the Apache Kafka Binary Distribution

Usually, the Kafka versions are compatible up to the minor version (for example, using the tools on the binary distribution of Kafka 2.0 usually works with other Kafka 2.X). If there is no particular reason against it, you should use the 2.12-XXX versions (Scala 2.12).

1. Download the Apache Kafka binary distribution at https://www.apache.org/dyn/closer.cgi?path=/kafka/.

   Beware of not downloading the -src.tgz-file which only contains the source code.

2. Uncompress the .tgz-file.

3. Save it in the desired location on your local machine.

   Example

   Assuming Linux/MacOS, you can download, uncompress and move the .tgz-file with the following commands:

   wget http://mirror.vorboss.net/apache/kafka/2.1.0/kafka_2.12-2.1.0.tgz -O /tmp/kafka_2.12-2.1.0.tgz
   tar -xzf /tmp/kafka_2.12-2.1.0.tgz -C /tmp
   rm /tmp/kafka_2.12-2.1.0.tgz
   mv /tmp/kafka_2.12-2.1.0 ~/kafka

   Now you should have your binary Kafka distribution on your home folder under the folder kafka.

4. In the desired location on your local machine, set up a JAAS (Java Authentication and Authorization Service) configuration file with the jaas.conf-extension. It contains the credentials and will be used when connecting to Kafka. Mapp will provide the credentials.

   This is a sensitive file so please set proper file permissions for the configuration file, as it contains the user and the password that will be used when connecting to Mapp's streaming endpoint.

5. Include the following in the jaas.conf-file:

   KafkaClient {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="myuser"
    password="mypassword";
   };

6. Substitute <username> and <password> by credentials provided by Mapp.

7. Export the environment variable KAFKA_OPTS from the path directory:

   export KAFKA_OPTS="-Djava.security.auth.login.config=/path/to/jaas.conf"

8. Substitute the path to your jaas.conf-file.