I'm trying to set up a Kafka broker with the SASL_SSL security protocol and the PLAIN mechanism for authentication. Below is my docker-compose configuration:
services:
zookeeper:
image: confluentinc/cp-zookeeper:7.4.4
container_name: zookeeper
ports:
- "2181:2181"
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/zookeeper/zookeeper_jaas.conf"
ZOOKEEPER_AUTH_PROVIDER_1: org.apache.zookeeper.server.auth.SASLAuthenticationProvider
volumes:
- ./secrets:/etc/kafka/secrets:ro
- ./secrets/zookeeper_jaas.conf:/etc/zookeeper/zookeeper_jaas.conf:ro
networks:
- kafka_network
kafka:
image: confluentinc/cp-kafka:7.4.4
user: "0"
container_name: kafka
ports:
- "9092:9092"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_LISTENERS: SASL_SSL://kafka:9093
KAFKA_ADVERTISED_LISTENERS: SASL_SSL://kafka:9093
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: SASL_SSL:SASL_SSL
KAFKA_SSL_KEYSTORE_FILENAME: kafka.keystore.jks
KAFKA_SSL_KEYSTORE_CREDENTIALS: keystore_creds
KAFKA_SSL_KEY_CREDENTIALS: key_creds
KAFKA_SSL_TRUSTSTORE_FILENAME: kafka.truststore.jks
KAFKA_SSL_TRUSTSTORE_CREDENTIALS: truststore_creds
KAFKA_SSL_CLIENT_AUTH: required
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
KAFKA_SASL_MECHANISMS: PLAIN
KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
KAFKA_SECURITY_PROTOCOL: SASL_SSL
KAFKA_AUTHORIZER_CLASS_NAME: "kafka.security.authorizer.AclAuthorizer"
KAFKA_SUPER_USERS: "User:admin"
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
CONFLUENT_SUPPORT_METRICS_ENABLE: 'false'
KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/kafka/secrets/kafka_server_jaas.conf"
KAFKA_INTER_BROKER_LISTENER_NAME: SASL_SSL
KAFKA_HOST_NAME: kafka
volumes:
- ./secrets:/etc/kafka/secrets:ro
depends_on:
- zookeeper
networks:
- kafka_network
networks:
kafka_network:
driver: bridge
I've also created the following JAAS files:
zookeeper_jaas.conf
:
Server {
org.apache.zookeeper.server.auth.DigestLoginModule required
username="admin"
password="admin_password"
user_admin="admin_password";
};
Client {
org.apache.zookeeper.server.auth.DigestLoginModule required
username="admin"
password="admin_password";
};
kafka_server_jaas.conf
:
KafkaServer {
org.apache.kafkamon.security.plain.PlainLoginModule required
username="admin"
password="admin_password";
};
Client {
org.apache.kafkamon.security.plain.PlainLoginModule required
username="admin"
password="admin_password";
};
When I start the containers, the Kafka broker fails with the following error in the logs:
ERROR [Controller id=1, targetBrokerId=1] Connection to node 1 (kafka/192.168.160.3:9093) failed authenticati
on due to: Authentication failed: Invalid username or password (org.apache.kafka.clients.NetworkClient)
[2025-01-17 13:45:50,896] WARN [RequestSendThread controllerId=1] Controller 1's connection to broker kafka:9093 (id: 1 rack: null) was
unsuccessful (kafka.controller.RequestSendThread)
org.apache.kafkamon.errors.SaslAuthenticationException: Authentication failed: Invalid username or password
[2025-01-17 13:45:50,896] INFO [Controller id=1, targetBrokerId=1] Client requested connection close from node 1 (org.apache.kafka.clie
nts.NetworkClient)
What might be causing the "Invalid username or password" error, and how can I resolve it?