I'm trying to configure Kafka kluster using Docker-compose with 2 options:
- Connect Spring Boot app inside the docker network in the future
- Connect Spring Boot app outside the docker network using localhost.
And my issue is that i can't establish connection from localy running application. It seems that it's trying to connect using kafka/kafka2 but not localhost. When lounching appllication i got repetitive logs:
2025-03-23T12:07:07.637+03:00 INFO 17988 --- [plummers-service] [ main] o.a.k.c.t.i.KafkaMetricsCollector : initializing Kafka metrics collector
2025-03-23T12:07:07.643+03:00 INFO 17988 --- [plummers-service] [ main] o.a.kafkamon.utils.AppInfoParser : Kafka version: 3.8.1
2025-03-23T12:07:07.643+03:00 INFO 17988 --- [plummers-service] [ main] o.a.kafkamon.utils.AppInfoParser : Kafka commitId: 70d6ff42debf7e17
2025-03-23T12:07:07.643+03:00 INFO 17988 --- [plummers-service] [ main] o.a.kafkamon.utils.AppInfoParser : Kafka startTimeMs: 1742720827643
2025-03-23T12:07:07.647+03:00 INFO 17988 --- [plummers-service] [ main] o.a.k.c.c.internals.LegacyKafkaConsumer : [Consumer clientId=consumer-plummers-service-8, groupId=plummers-service] Subscribed to topic(s): topic1
2025-03-23T12:07:07.667+03:00 INFO 17988 --- [plummers-service] [ main] c.m.p.PlummersServiceApplication : Started PlummersServiceApplication in 7.448 seconds (process running for 8.612)
2025-03-23T12:07:07.958+03:00 INFO 17988 --- [plummers-service] [ntainer#1-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-6, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:07.958+03:00 INFO 17988 --- [plummers-service] [ntainer#1-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-5, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:07.958+03:00 INFO 17988 --- [plummers-service] [ntainer#0-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-3, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:07.958+03:00 INFO 17988 --- [plummers-service] [ntainer#1-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-7, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:07.958+03:00 INFO 17988 --- [plummers-service] [ntainer#0-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-1, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:07.959+03:00 INFO 17988 --- [plummers-service] [ntainer#0-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-2, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:07.964+03:00 WARN 17988 --- [plummers-service] [ntainer#0-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-1, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:07.964+03:00 WARN 17988 --- [plummers-service] [ntainer#0-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-3, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:07.964+03:00 WARN 17988 --- [plummers-service] [ntainer#1-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-6, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:07.965+03:00 WARN 17988 --- [plummers-service] [ntainer#1-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-5, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:07.964+03:00 WARN 17988 --- [plummers-service] [ntainer#0-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-2, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:07.965+03:00 WARN 17988 --- [plummers-service] [ntainer#1-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-7, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:07.965+03:00 WARN 17988 --- [plummers-service] [ntainer#1-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-5, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:07.965+03:00 WARN 17988 --- [plummers-service] [ntainer#1-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-6, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:07.965+03:00 WARN 17988 --- [plummers-service] [ntainer#0-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-2, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:07.965+03:00 WARN 17988 --- [plummers-service] [ntainer#1-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-7, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:07.965+03:00 WARN 17988 --- [plummers-service] [ntainer#0-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-1, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:07.965+03:00 WARN 17988 --- [plummers-service] [ntainer#0-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-3, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:08.173+03:00 INFO 17988 --- [plummers-service] [ntainer#1-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-6, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:08.173+03:00 INFO 17988 --- [plummers-service] [ntainer#1-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-5, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:08.173+03:00 WARN 17988 --- [plummers-service] [ntainer#1-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-6, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:08.173+03:00 WARN 17988 --- [plummers-service] [ntainer#1-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-6, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:08.173+03:00 INFO 17988 --- [plummers-service] [ntainer#0-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-1, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:08.173+03:00 WARN 17988 --- [plummers-service] [ntainer#0-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-1, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:08.173+03:00 WARN 17988 --- [plummers-service] [ntainer#0-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-1, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:08.174+03:00 WARN 17988 --- [plummers-service] [ntainer#1-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-5, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:08.173+03:00 INFO 17988 --- [plummers-service] [ntainer#0-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-2, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:08.175+03:00 INFO 17988 --- [plummers-service] [ntainer#1-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-7, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:08.175+03:00 WARN 17988 --- [plummers-service] [ntainer#1-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-5, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:08.176+03:00 WARN 17988 --- [plummers-service] [ntainer#1-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-7, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:08.175+03:00 WARN 17988 --- [plummers-service] [ntainer#0-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-2, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:08.175+03:00 INFO 17988 --- [plummers-service] [ntainer#0-3-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-4, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:08.177+03:00 WARN 17988 --- [plummers-service] [ntainer#1-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-7, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:08.177+03:00 WARN 17988 --- [plummers-service] [ntainer#0-1-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-2, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:08.177+03:00 INFO 17988 --- [plummers-service] [ntainer#0-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-3, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:08.177+03:00 WARN 17988 --- [plummers-service] [ntainer#0-3-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-4, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:08.178+03:00 WARN 17988 --- [plummers-service] [ntainer#0-3-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-4, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:08.178+03:00 INFO 17988 --- [plummers-service] [ntainer#1-3-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-8, groupId=plummers-service] Node -2 disconnected.
2025-03-23T12:07:08.178+03:00 WARN 17988 --- [plummers-service] [ntainer#1-3-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-8, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:08.178+03:00 WARN 17988 --- [plummers-service] [ntainer#1-3-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-8, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:08.178+03:00 WARN 17988 --- [plummers-service] [ntainer#0-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-3, groupId=plummers-service] Connection to node -2 (localhost/127.0.0.1:29094) could not be established. Node may not be available.
2025-03-23T12:07:08.178+03:00 WARN 17988 --- [plummers-service] [ntainer#0-2-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-3, groupId=plummers-service] Bootstrap broker localhost:29094 (id: -2 rack: null) disconnected
2025-03-23T12:07:08.199+03:00 INFO 17988 --- [plummers-service] [ntainer#1-2-C-1] .apache.kafka.clients.Metadata : [Consumer clientId=consumer-plummers-service-7, groupId=plummers-service] Cluster ID: 3O2HYFDmSN--K19sk_XNPw
2025-03-23T12:07:08.199+03:00 INFO 17988 --- [plummers-service] [ntainer#0-0-C-1] .apache.kafka.clients.Metadata : [Consumer clientId=consumer-plummers-service-1, groupId=plummers-service] Cluster ID: 3O2HYFDmSN--K19sk_XNPw
2025-03-23T12:07:08.199+03:00 INFO 17988 --- [plummers-service] [ntainer#0-2-C-1] .apache.kafka.clients.Metadata : [Consumer clientId=consumer-plummers-service-3, groupId=plummers-service] Cluster ID: 3O2HYFDmSN--K19sk_XNPw
2025-03-23T12:07:08.199+03:00 INFO 17988 --- [plummers-service] [ntainer#1-1-C-1] .apache.kafka.clients.Metadata : [Consumer clientId=consumer-plummers-service-6, groupId=plummers-service] Cluster ID: 3O2HYFDmSN--K19sk_XNPw
2025-03-23T12:07:08.199+03:00 INFO 17988 --- [plummers-service] [ntainer#0-3-C-1] .apache.kafka.clients.Metadata : [Consumer clientId=consumer-plummers-service-4, groupId=plummers-service] Cluster ID: 3O2HYFDmSN--K19sk_XNPw
2025-03-23T12:07:08.199+03:00 INFO 17988 --- [plummers-service] [ntainer#1-3-C-1] .apache.kafka.clients.Metadata : [Consumer clientId=consumer-plummers-service-8, groupId=plummers-service] Cluster ID: 3O2HYFDmSN--K19sk_XNPw
2025-03-23T12:07:08.199+03:00 INFO 17988 --- [plummers-service] [ntainer#0-1-C-1] .apache.kafka.clients.Metadata : [Consumer clientId=consumer-plummers-service-2, groupId=plummers-service] Cluster ID: 3O2HYFDmSN--K19sk_XNPw
2025-03-23T12:07:08.199+03:00 INFO 17988 --- [plummers-service] [ntainer#1-0-C-1] .apache.kafka.clients.Metadata : [Consumer clientId=consumer-plummers-service-5, groupId=plummers-service] Cluster ID: 3O2HYFDmSN--K19sk_XNPw
2025-03-23T12:07:08.201+03:00 INFO 17988 --- [plummers-service] [ntainer#1-3-C-1] o.a.k.c.c.internals.ConsumerCoordinator : [Consumer clientId=consumer-plummers-service-8, groupId=plummers-service] Discovered group coordinator kafka2:29092 (id: 2147483645 rack: null)
2025-03-23T12:07:10.472+03:00 WARN 17988 --- [plummers-service] [ntainer#1-0-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-5, groupId=plummers-service] Error connecting to node kafka2:29092 (id: 2 rack: null)
java.UnknownHostException: kafka2
at java.base/java.InetAddress$CachedAddresses.get(InetAddress.java:952) ~[na:na]
at java.base/java.InetAddress$NameServiceAddresses.get(InetAddress.java:1040) ~[na:na]
at java.base/java.InetAddress.getAllByName0(InetAddress.java:1663) ~[na:na]
at java.base/java.InetAddress.getAllByName(InetAddress.java:1528) ~[na:na]
at .apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:124) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.resolveAddresses(ClusterConnectionStates.java:536) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:511) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:466) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:1070) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.NetworkClient.ready(NetworkClient.java:320) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.ConsumerNetworkClient.trySend(ConsumerNetworkClient.java:514) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:271) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:231) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:289) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:263) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.ConsumerCoordinator.coordinatorUnknownAndUnreadySync(ConsumerCoordinator.java:450) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:482) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.LegacyKafkaConsumer.updateAssignmentMetadataIfNeeded(LegacyKafkaConsumer.java:652) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.LegacyKafkaConsumer.poll(LegacyKafkaConsumer.java:611) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.LegacyKafkaConsumer.poll(LegacyKafkaConsumer.java:591) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:874) ~[kafka-clients-3.8.1.jar:na]
at .springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollConsumer(KafkaMessageListenerContainer.java:1692) ~[spring-kafka-3.3.4.jar:3.3.4]
at .springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doPoll(KafkaMessageListenerContainer.java:1667) ~[spring-kafka-3.3.4.jar:3.3.4]
at .springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1445) ~[spring-kafka-3.3.4.jar:3.3.4]
at .springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1335) ~[spring-kafka-3.3.4.jar:3.3.4]
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:1589) ~[na:na]
2025-03-23T12:07:10.472+03:00 WARN 17988 --- [plummers-service] [ntainer#0-3-C-1] .apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-plummers-service-4, groupId=plummers-service] Error connecting to node kafka:29092 (id: 1 rack: null)
java.UnknownHostException: kafka
at java.base/java.InetAddress$CachedAddresses.get(InetAddress.java:952) ~[na:na]
at java.base/java.InetAddress$NameServiceAddresses.get(InetAddress.java:1040) ~[na:na]
at java.base/java.InetAddress.getAllByName0(InetAddress.java:1663) ~[na:na]
at java.base/java.InetAddress.getAllByName(InetAddress.java:1528) ~[na:na]
at .apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:124) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.resolveAddresses(ClusterConnectionStates.java:536) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:511) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:466) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:1070) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.NetworkClient.access$800(NetworkClient.java:76) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.NetworkClient$TelemetrySender.maybeUpdate(NetworkClient.java:1335) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.NetworkClient$TelemetrySender.maybeUpdate(NetworkClient.java:1304) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.NetworkClient.poll(NetworkClient.java:593) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:281) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:231) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:289) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:263) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.ConsumerCoordinator.coordinatorUnknownAndUnreadySync(ConsumerCoordinator.java:450) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:482) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.LegacyKafkaConsumer.updateAssignmentMetadataIfNeeded(LegacyKafkaConsumer.java:652) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.LegacyKafkaConsumer.poll(LegacyKafkaConsumer.java:611) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.internals.LegacyKafkaConsumer.poll(LegacyKafkaConsumer.java:591) ~[kafka-clients-3.8.1.jar:na]
at .apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:874) ~[kafka-clients-3.8.1.jar:na]
at .springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollConsumer(KafkaMessageListenerContainer.java:1692) ~[spring-kafka-3.3.4.jar:3.3.4]
at .springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doPoll(KafkaMessageListenerContainer.java:1667) ~[spring-kafka-3.3.4.jar:3.3.4]
at .springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1445) ~[spring-kafka-3.3.4.jar:3.3.4]
at .springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1335) ~[spring-kafka-3.3.4.jar:3.3.4]
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:1589) ~[na:na]
Here is my docker-compose:
zookeeper:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ports:
- 22181:2181
kafka:
image: confluentinc/cp-kafka:latest
depends_on:
- zookeeper
ports:
- 29093:29093
hostname: kafka
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:29092,EXTERNAL://localhost:29093
KAFKA_LISTENERS: INTERNAL://0.0.0.0:29092,EXTERNAL://0.0.0.0:29093
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 2
kafka2:
image: confluentinc/cp-kafka:latest
depends_on:
- zookeeper
ports:
- 29094:29094
hostname: kafka2
environment:
KAFKA_BROKER_ID: 2
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka2:29092,EXTERNAL://localhost:29094
KAFKA_LISTENERS: INTERNAL://0.0.0.0:29092,EXTERNAL://0.0.0.0:29094
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 2
kafka-ui:
image: provectuslabs/kafka-ui
container_name: kafka-ui
ports:
- 8090:8080
restart: always
environment:
- KAFKA_CLUSTERS_0_NAME=local
- KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:29092,kafka2:29092
- KAFKA_CLUSTERS_0_ZOOKEEPER=zookeeper:2181
links:
- kafka
- kafka2
- zookeeper
And application config:
config:
host: localhost
port: 8071
spring:
application:
name: plummers-service
config:
import: configserver:http://${config.host}:${config.port}
cloud:
config:
fail-fast: true
enabled: true
kafka:
bootstrap-servers: localhost:29093,localhost:29094
listener:
ack-mode: record
producer:
client-id: ${spring.application.name}
key-serializer: .apache.kafkamon.serialization.StringSerializer
value-serializer: com.meridian.plummersservice.config.serialize.DltMessageSerializer
retries: 3
consumer:
group-id: ${spring.application.name}
autoOffsetReset: earliest
keyDeserializer: .springframework.kafka.support.serializer.ErrorHandlingDeserializer
valueDeserializer: .springframework.kafka.support.serializer.ErrorHandlingDeserializer
properties:
spring.json.value.default.type: com.meridian.plummersservice.dto.kafka.TestMessage
spring.deserializer.key.delegate.class: .apache.kafkamon.serialization.StringDeserializer
spring.deserializer.value.delegate.class: .springframework.kafka.support.serializer.JsonDeserializer
kafka:
topics:
test-topic: topic1