- 1 Minute to read
- Print
- DarkLight
Prepare the Apache Kafka Binary Distribution
- 1 Minute to read
- Print
- DarkLight
Usually, the Kafka versions are compatible up to the minor version (for example, using the tools on the binary distribution of Kafka 2.0 usually works with other Kafka 2.X). If there is no particular reason against it, you should use the 2.12-XXX versions (Scala 2.12).
Download the Apache Kafka binary distribution at https://www.apache.org/dyn/closer.cgi?path=/kafka/.
Beware of not downloading the
-src.tgz
-file which only contains the source code.Uncompress the
.tgz
-file.Save it in the desired location on your local machine.
Example
Assuming Linux/MacOS, you can download, uncompress and move the
.tgz
-file with the following commands:wget http://mirror.vorboss.net/apache/kafka/2.1.0/kafka_2.12-2.1.0.tgz -O /tmp/kafka_2.12-2.1.0.tgz tar -xzf /tmp/kafka_2.12-2.1.0.tgz -C /tmp rm /tmp/kafka_2.12-2.1.0.tgz mv /tmp/kafka_2.12-2.1.0 ~/kafka
Now you should have your binary Kafka distribution on your home folder under the folder
kafka
.In the desired location on your local machine, set up a JAAS (Java Authentication and Authorization Service) configuration file with the
jaas.conf
-extension. It contains the credentials and will be used when connecting to Kafka. Mapp will provide the credentials.This is a sensitive file so please set proper file permissions for the configuration file, as it contains the user and the password that will be used when connecting to Mapp's streaming endpoint.
Include the following in the
jaas.conf
-file:KafkaClient { org.apache.kafka.common.security.scram.ScramLoginModule required username="myuser" password="mypassword"; };
Substitute <username> and <password> by credentials provided by Mapp.
Export the environment variable
KAFKA_OPTS
from the path directory:export KAFKA_OPTS="-Djava.security.auth.login.config=/path/to/jaas.conf"
Substitute the path to your
jaas.conf
-file.