I'm running Kafka with KRaft in docker-compose with the following configuration:

services:
  kafka:
    image: apache/kafka:4.0.0
    container_name: kafka
    restart: always
    ports:
      - "9092:9092"
      - "9093:9093"
      - "9094:9094"
    volumes:
      - kafka_data:/var/lib/kafka/data
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093,EXTERN://:9094
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,EXTERN://host.docker.internal:9094
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,EXTERN:PLAINTEXT
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@localhost:9093

Producing messages in topics works fine, but when I try to consume from them something strange happens.

For example: I have topic test with 1 partition. I send messages into this topic (from console-producer). When I'm trying to consume messages by following command:

./kafka-console-consumer.sh --bootstrap-server="host.docker.internal:9094" --topic="test" --from-beginning

it doesn't consume anything at all. But if I specify the partition I get all the messages and everything works fine:

./kafka-console-consumer.sh --bootstrap-server="host.docker.internal:9094" --topic="test" --from-beginning --partition=0

I was trying to figure out what's happening and noticed this thing: if I start consuming from the topic with specific consumer group id:

./kafka-console-consumer.sh --bootstrap-server="host.docker.internal:9094" --topic="test" --from-beginning --group="my-group"

then I try to get the list of consumer groups and I got nothing!

./kafka-consumer-groups.sh --list --bootstrap-server="host.docker.internal:9094" # Prints nothing

I tried to consume messages in golang kafka client and everything works the same - if I specify partition - I get messages, if I don't - I don't get messages. It seems like Kafka can't create consumer groups. I'm new to Kafka and maybe I don't understand something.