# Create SSL Certificates for Kafka Servers When creating the SSL certs kindly use `masKaf20212ndJan609Pm` for any password field 1. Log on to kafka 1 (5.9.8.66). Move to location /var/www/html/kafka and creata a temp folder for the SSL creation **run command** ``` cd /var/www/html/kafka mkdir temp cd temp ``` 2. Start SSL creation process by generating keystore for each server **run keytool command for master1** ``` keytool -keyalg RSA -keystore kafka.master1.keystore.jks -alias master1 -validity 365 -genkey -storepass masKaf20212ndJan609Pm -keypass masKaf20212ndJan609Pm -ext SAN=DNS:static.66.8.9.5.clients.your-server.de ``` **It will ask some questions before generating each keystore. First of which will be** >> What is your first and last name? >> static.66.8.9.5.clients.your-server.de **run keytool command for master2** ``` keytool -keyalg RSA -keystore kafka.master2.keystore.jks -alias master2 -validity 365 -genkey -storepass masKaf20212ndJan609Pm -keypass masKaf20212ndJan609Pm -ext SAN=DNS:static.79.14.243.136.clients.your-server.de ``` #It will ask some questions before generating each keystore. First of which will be — >> What is your first and last name? >> static.79.14.243.136.clients.your-server.de **run keytool command for worker2** ``` keytool -keyalg RSA -keystore kafka.worker2.keystore.jks -alias worker2 -validity 365 -genkey -storepass masKaf20212ndJan609Pm -keypass masKaf20212ndJan609Pm -ext SAN=DNS:static.148.126.12.49.clients.your-server.de ``` #It will ask some questions before generating each keystore. First of which will be — >> What is your first and last name? >> static.148.126.12.49.clients.your-server.de **run keytool command for kafka3** ``` keytool -keyalg RSA -keystore kafka.kafka3.keystore.jks -alias kafka3 -validity 365 -genkey -storepass masKaf20212ndJan609Pm -keypass masKaf20212ndJan609Pm -ext SAN=DNS:static.171.45.217.95.clients.your-server.de ``` #It will ask some questions before generating each keystore. First of which will be — >> What is your first and last name? >> static.171.45.217.95.clients.your-server.de Answer the other questions as you like and you keystores for each server is ready. 3. Create Certificate Authority - in PEM format ``` openssl req -new -x509 -keyout ca-key -out ca-cert -days 365 ``` // Pass masKaf20212ndJan609Pm to any password field 4. Add this CA to client & Broker truststore. Now, broker can trust the certificates signed by this CA. These JKS truststores containing public key of CA ``` keytool -keystore kafka.client.truststore.jks -alias CARoot -import -file ca-cert ``` // Pass masKaf20212ndJan609Pm to any password field ``` keytool -keystore kafka.server.truststore.jks -alias CARoot -importcert -file ca-cert ``` // Pass masKaf20212ndJan609Pm to any password field 5. To sign the broker certificates by CA cert and key, we first convert JKS to PEM thus creating Certificate Signing Request(CSR files) ``` keytool -keystore kafka.master1.keystore.jks -alias master1 -certreq -file cert-file1 keytool -keystore kafka.master2.keystore.jks -alias master2 -certreq -file cert-file2 keytool -keystore kafka.worker2.keystore.jks -alias worker2 -certreq -file cert-file3 keytool -keystore kafka.kafka3.keystore.jks -alias kafka3 -certreq -file cert-file4 ``` 6. Sign the CSRs with the CA cert and key finally creating signed certificates in PEM format ``` openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file1 -out cert-signed1 -days 365 -CAcreateserial -passin pass:masKaf20212ndJan609Pm openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file2 -out cert-signed2 -days 365 -CAcreateserial -passin pass:masKaf20212ndJan609Pm openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file3 -out cert-signed3 -days 365 -CAcreateserial -passin pass:masKaf20212ndJan609Pm openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file4 -out cert-signed4 -days 365 -CAcreateserial -passin pass:masKaf20212ndJan609Pm ``` 7. Public key of CA is added to the broker keystores ``` keytool -keystore kafka.master1.keystore.jks -alias CARoot -import -file ca-cert keytool -keystore kafka.master2.keystore.jks -alias CARoot -import -file ca-cert keytool -keystore kafka.worker2.keystore.jks -alias CARoot -import -file ca-cert keytool -keystore kafka.kafka3.keystore.jks -alias CARoot -import -file ca-cert ``` 8. Public key of signed Certificates is added to the broker keystores ``` keytool -keystore kafka.master1.keystore.jks -alias master1 -import -file cert-signed1 keytool -keystore kafka.master2.keystore.jks -alias master2 -import -file cert-signed2 keytool -keystore kafka.worker2.keystore.jks -alias worker2 -import -file cert-signed3 keytool -keystore kafka.kafka3.keystore.jks -alias kafka3 -import -file cert-signed4 ``` // Pass masKaf20212ndJan609Pm to any password field 9. Create .pem files used by filebeat and other services ``` keytool -exportcert -alias CARoot -keystore kafka.server.truststore.jks -rfc -file my_key.pem -storepass "masKaf20212ndJan609Pm" -keypass "masKaf20212ndJan609Pm" -noprompt ``` 10. Backup old certs and Transfer files to appropriate server and locations. the .pem Transfer is sent to all servers with filebeat. The .pem is stored at /var/www/html/kafka/certs ``` mv /var/www/html/kafka/certs /var/www/html/kafka/certs-bk- mkdir /var/www/html/kafka/certs cp /var/www/html/kafka/temp/kafka.master1.keystore.jks /var/www/html/kafka/certs/kafka.master1.keystore.jks cp /var/www/html/kafka/temp/kafka.server.truststore.jks /var/www/html/kafka/certs/kafka.server.truststore.jks scp -P22 -i kafka_2 /var/www/html/kafka/temp/kafka.master2.keystore.jks root@136.243.14.79:/var/www/html/kafka/certs scp -P22 -i kafka_2 /var/www/html/kafka/temp/kafka.server.truststore.jks root@136.243.14.79:/var/www/html/kafka/certs scp -P22 /var/www/html/kafka/certs/kafka.server.truststore.jks root@148.251.12.91:/var/www/html/kafka/certs scp -P22 /var/www/html/kafka/certs/my_key.pem root@148.251.12.91:/var/www/html/kafka/certs scp -P22 -i kafka_3 /var/www/html/kafka/temp/kafka.kafka3.keystore.jks root@95.217.45.171:/var/www/html/kafka/certs scp -P22 -i kafka_3 /var/www/html/kafka/temp/kafka.server.truststore.jks root@95.217.45.171:/var/www/html/kafka/certs scp -P22 -i prometheous /var/www/html/kafka/certs/kafka.server.truststore.jks root@95.216.26.173:/var/www/html/kafka/certs scp -P2108 /var/www/html/kafka/certs/kafka.server.truststore.jks abraham@95.216.10.31:/var/www/html/kafka/certs scp -P22 /var/www/html/kafka/certs/kafka.server.truststore.jks abraham@144.76.58.179:/var/www/html/kafka/certs scp -P22 /var/www/html/kafka/certs/kafka.server.truststore.jks abraham@5.9.61.79:/var/www/html/kafka_certs ``` 11. Restart kafka, filebeat and supervisors managing producer and consumer scripts 1. Stop kafka on all servers ``` systemctl stop confluent-zookeeper systemctl stop confluent-server systemctl stop kafka-connect.service systemctl stop confluent-schema-registry systemctl stop confluent-ksqldb ``` 2. Start kafka on all servers ``` systemctl start confluent-zookeeper systemctl start confluent-server systemctl start kafka-connect.service systemctl start confluent-schema-registry systemctl start confluent-ksqldb ``` 3. Restart filebeat on all server ``` systemctl restart filebeat ``` 4. Update group.id in scripts and restart supervisor 5. Ensure all services are running as expected. kindly note it may take a few hours for new updates to catch up if the volumes in the queue is large.