Skip to content

KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS with domain name issue #4537

Open
@zappee

Description

@zappee

Issue submitter TODO list

  • I've looked up my issue in FAQ
  • I've searched for an already existing issues here
  • I've tried running master-labeled docker image and the issue still persists there
  • I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

I am running kafka-ui in docker using the following compose file:

    kafka-ui:
        image: ghcr.io/kafbat/kafka-ui:v1.2.0
        restart: unless-stopped
        container_name: kafka-ui.${DOMAIN_NAME}
        hostname: kafka-ui.${DOMAIN_NAME}
        read_only: true
        environment:
            KAFKA_CLUSTERS_0_NAME: ${KAFKA_CLUSTER_NAME}
            KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka-4.${DOMAIN_NAME}:9092,kafka-5.${DOMAIN_NAME}:9092,kafka-6.${DOMAIN_NAME}:9092,kafka-7.${DOMAIN_NAME}:9092
            KAFKA_CLUSTERS_0_METRICS_PORT: 9997
            KAFKA_CLUSTERS_0_METRICS_TYPE: JMX
            DYNAMIC_CONFIG_ENABLED: true
        ports:
            - "25280:8080"
        depends_on:
            kafka-4:
                condition: service_healthy
            kafka-5:
                condition: service_healthy
            kafka-6:
                condition: service_healthy
            kafka-7:
                condition: service_healthy

I have multiple bootstrap address with domain name (hello.com) , but as I see from the log, only the first address was picked up properly:


2025-06-04 17:48:59,743 WARN  [main] i.k.u.u.DynamicConfigOperations: Dynamic config file /etc/kafkaui/dynamic_config.yaml doesnt exist or not readable
2025-06-04 17:48:59,751 INFO  [main] i.k.u.KafkaUiApplication: Starting KafkaUiApplication vv1.2.0 using Java 21.0.6 with PID 1 (/api.jar started by kafkaui in /)
2025-06-04 17:48:59,752 DEBUG [main] i.k.u.KafkaUiApplication: Running with Spring Boot v3.4.3, Spring v6.2.3
2025-06-04 17:48:59,753 INFO  [main] i.k.u.KafkaUiApplication: No active profile set, falling back to 1 default profile: "default"
2025-06-04 17:49:05,968 DEBUG [main] i.k.u.s.SerdesInitializer: Configuring serdes for cluster kafka-cluster
2025-06-04 17:49:07,359 INFO  [main] o.s.b.a.e.w.EndpointLinksResolver: Exposing 3 endpoints beneath base path '/actuator'
2025-06-04 17:49:07,839 WARN  [main] i.k.u.c.a.DisabledAuthSecurityConfig: Authentication is disabled. Access will be unrestricted.
2025-06-04 17:49:09,705 INFO  [main] o.s.b.w.e.n.NettyWebServer: Netty started on port 8080 (http)
2025-06-04 17:49:09,787 INFO  [main] i.k.u.KafkaUiApplication: Started KafkaUiApplication in 11.141 seconds (process running for 12.239)
2025-06-04 17:49:10,398 DEBUG [parallel-2] i.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: kafka-cluster
2025-06-04 17:49:10,431 INFO  [boundedElastic-1] o.a.k.c.a.AdminClientConfig: AdminClientConfig values: 
        auto.include.jmx.reporter = true
        bootstrap.controllers = []
        bootstrap.servers = [kafka-4.hello.com:9092, kafka-5.hello.com:9092, kafka-6.hello.com:9092, kafka-7.hello.com:9092]
        client.dns.lookup = use_all_dns_ips

...

025-06-04 17:49:10,602 INFO  [boundedElastic-1] o.a.k.c.u.AppInfoParser: Kafka version: 7.9.0-ccs
2025-06-04 17:49:10,603 INFO  [boundedElastic-1] o.a.k.c.u.AppInfoParser: Kafka commitId: ebe6df624d6bc758
2025-06-04 17:49:10,603 INFO  [boundedElastic-1] o.a.k.c.u.AppInfoParser: Kafka startTimeMs: 1749059350596
2025-06-04 17:49:12,001 DEBUG [boundedElastic-1] i.k.u.s.m.JmxMetricsRetriever: Collection JMX metrics for service:jmx:rmi:///jndi/rmi://kafka-4.hello.com:9997/jmxrmi
2025-06-04 17:49:12,002 DEBUG [boundedElastic-2] i.k.u.s.m.JmxMetricsRetriever: Collection JMX metrics for service:jmx:rmi:///jndi/rmi://kafka-5:9997/jmxrmi
2025-06-04 17:49:12,008 DEBUG [boundedElastic-4] i.k.u.s.m.JmxMetricsRetriever: Collection JMX metrics for service:jmx:rmi:///jndi/rmi://kafka-7:9997/jmxrmi
2025-06-04 17:49:12,009 DEBUG [boundedElastic-3] i.k.u.s.m.JmxMetricsRetriever: Collection JMX metrics for service:jmx:rmi:///jndi/rmi://kafka-6:9997/jmxrmi
2025-06-04 17:49:13,963 DEBUG [boundedElastic-3] i.k.u.s.m.JmxMetricsRetriever: 734 metrics collected for service:jmx:rmi:///jndi/rmi://kafka-6:9997/jmxrmi
2025-06-04 17:49:14,209 DEBUG [boundedElastic-1] i.k.u.s.m.JmxMetricsRetriever: 734 metrics collected for service:jmx:rmi:///jndi/rmi://kafka-4.hello.com:9997/jmxrmi
2025-06-04 17:49:14,378 DEBUG [boundedElastic-2] i.k.u.s.m.JmxMetricsRetriever: 735 metrics collected for service:jmx:rmi:///jndi/rmi://kafka-5:9997/jmxrmi
2025-06-04 17:49:14,539 DEBUG [boundedElastic-4] i.k.u.s.m.JmxMetricsRetriever: 734 metrics collected for service:jmx:rmi:///jndi/rmi://kafka-7:9997/jmxrmi
2025-06-04 17:49:14,558 DEBUG [boundedElastic-4] i.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: kafka-cluster

Expected behavior

Use the FQDN always:


2025-06-04 18:02:39,779 DEBUG [boundedElastic-6] i.k.u.s.m.JmxMetricsRetriever: Collection JMX metrics for service:jmx:rmi:///jndi/rmi://kafka-6.hello.com:9997/jmxrmi
2025-06-04 18:02:39,779 DEBUG [boundedElastic-5] i.k.u.s.m.JmxMetricsRetriever: Collection JMX metrics for service:jmx:rmi:///jndi/rmi://kafka-5.hello.com:9997/jmxrmi
2025-06-04 18:02:39,779 DEBUG [boundedElastic-2] i.k.u.s.m.JmxMetricsRetriever: Collection JMX metrics for service:jmx:rmi:///jndi/rmi://kafka-7.hello.com:9997/jmxrmi
2025-06-04 18:02:39,779 DEBUG [boundedElastic-3] i.k.u.s.m.JmxMetricsRetriever: Collection JMX metrics for service:jmx:rmi:///jndi/rmi://kafka-4.hello.com:9997/jmxrmi
2025-06-04 18:02:39,874 DEBUG [boundedElastic-5] i.k.u.s.m.JmxMetricsRetriever: 735 metrics collected for service:jmx:rmi:///jndi/rmi://kafka-5,hello.com:9997/jmxrmi
2025-06-04 18:02:39,877 DEBUG [boundedElastic-6] i.k.u.s.m.JmxMetricsRetriever: 734 metrics collected for service:jmx:rmi:///jndi/rmi://kafka-6.hello.com:9997/jmxrmi
2025-06-04 18:02:39,880 DEBUG [boundedElastic-3] i.k.u.s.m.JmxMetricsRetriever: 734 metrics collected for service:jmx:rmi:///jndi/rmi://kafka-4.hello.com:9997/jmxrmi
2025-06-04 18:02:39,884 DEBUG [boundedElastic-2] i.k.u.s.m.JmxMetricsRetriever: 732 metrics collected for service:jmx:rmi:///jndi/rmi://kafka-7.hello.com:9997/jmxrmi

Your installation details

Use the following docker compose file:

    kafka-1:
        image: docker.io/apache/kafka:4.0.0
        restart: unless-stopped
        container_name: kafka-1.${DOMAIN_NAME}
        hostname: kafka-1.${DOMAIN_NAME}
        environment:
            KAFKA_CLUSTER_ID: ${KAFKA_CLUSTER_ID}
            KAFKA_NODE_ID: 1
            KAFKA_PROCESS_ROLES: controller
            KAFKA_LISTENERS: CONTROLLER://:9093
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT
            KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
            KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka-1.${DOMAIN_NAME}:9093,2@kafka-2.${DOMAIN_NAME}:9093,3@kafka-3.${DOMAIN_NAME}:9093
            KAFKA_JMX_PORT: 9997
        healthcheck:
            test: /opt/kafka/bin/kafka-cluster.sh list-endpoints --bootstrap-controller localhost:9093
            interval: 60s
            timeout: 10s
            retries: 3
            start_period: 1s
            start_interval: 1s

    kafka-2:
        image: docker.io/apache/kafka:4.0.0
        restart: unless-stopped
        container_name: kafka-2.${DOMAIN_NAME}
        hostname: kafka-2.${DOMAIN_NAME}
        environment:
            KAFKA_CLUSTER_ID: ${KAFKA_CLUSTER_ID}
            KAFKA_NODE_ID: 2
            KAFKA_PROCESS_ROLES: controller
            KAFKA_LISTENERS: CONTROLLER://:9093
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT
            KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
            KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka-1.${DOMAIN_NAME}:9093,2@kafka-2.${DOMAIN_NAME}:9093,3@kafka-3.${DOMAIN_NAME}:9093
            KAFKA_JMX_PORT: 9997
        healthcheck:
            test: /opt/kafka/bin/kafka-cluster.sh list-endpoints --bootstrap-controller localhost:9093
            interval: 60s
            timeout: 10s
            retries: 3
            start_period: 1s
            start_interval: 1s

    kafka-3:
        image: docker.io/apache/kafka:4.0.0
        restart: unless-stopped
        container_name: kafka-3.${DOMAIN_NAME}
        hostname: kafka-3.${DOMAIN_NAME}
        environment:
            KAFKA_CLUSTER_ID: ${KAFKA_CLUSTER_ID}
            KAFKA_NODE_ID: 3
            KAFKA_PROCESS_ROLES: controller
            KAFKA_LISTENERS: CONTROLLER://:9093
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT
            KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
            KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka-1.${DOMAIN_NAME}:9093,2@kafka-2.${DOMAIN_NAME}:9093,3@kafka-3.${DOMAIN_NAME}:9093
            KAFKA_JMX_PORT: 9997
        healthcheck:
            test: /opt/kafka/bin/kafka-cluster.sh list-endpoints --bootstrap-controller localhost:9093
            interval: 60s
            timeout: 10s
            retries: 3
            start_period: 1s
            start_interval: 1s

    # ----- Kafka brokers -----
    kafka-4:
        image: docker.io/apache/kafka:4.0.0
        restart: unless-stopped
        container_name: kafka-4.${DOMAIN_NAME}
        hostname: kafka-4.${DOMAIN_NAME}
        environment:
            KAFKA_CLUSTER_ID: ${KAFKA_CLUSTER_ID}
            KAFKA_NODE_ID: 4
            KAFKA_PROCESS_ROLES: broker
            KAFKA_LISTENERS: INTERNAL://:9092,EXTERNAL://:9094
            KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka-4.${DOMAIN_NAME}:9092,EXTERNAL://${KAFKA_EXTERNAL_IP}:${KAFKA_BROKER_1_PORT}
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
            KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
            KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
            KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka-1.${DOMAIN_NAME}:9093,2@kafka-2.${DOMAIN_NAME}:9093,3@kafka-3.${DOMAIN_NAME}:9093
            KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
            KAFKA_LISTENER_NAME_EXTERNAL_PLAIN_SASL_JAAS_CONFIG: |
                org.apache.kafka.common.security.plain.PlainLoginModule required \
                username="${KAFKA_USERNAME}" \
                password="${KAFKA_PASSWORD}" \
                user_${KAFKA_USERNAME}="${KAFKA_PASSWORD}";
            KAFKA_JMX_PORT: 9997
        ports:
            - ${KAFKA_BROKER_1_PORT}:9094/tcp
        healthcheck:
            test: /opt/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092
            interval: 60s
            timeout: 10s
            retries: 3
            start_period: 1s
            start_interval: 1s
        depends_on:
            kafka-1:
                condition: service_healthy
            kafka-2:
                condition: service_healthy
            kafka-3:
                condition: service_healthy

    kafka-5:
        image: docker.io/apache/kafka:4.0.0
        restart: unless-stopped
        container_name: kafka-5.${DOMAIN_NAME}
        hostname: kafka-5.${DOMAIN_NAME}
        environment:
            KAFKA_CLUSTER_ID: ${KAFKA_CLUSTER_ID}
            KAFKA_NODE_ID: 5
            KAFKA_PROCESS_ROLES: broker
            KAFKA_LISTENERS: INTERNAL://:9092,EXTERNAL://:9094
            KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka-5:9092,EXTERNAL://${KAFKA_EXTERNAL_IP}:${KAFKA_BROKER_2_PORT}
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
            KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
            KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
            KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka-1.${DOMAIN_NAME}:9093,2@kafka-2.${DOMAIN_NAME}:9093,3@kafka-3.${DOMAIN_NAME}:9093
            KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
            KAFKA_LISTENER_NAME_EXTERNAL_PLAIN_SASL_JAAS_CONFIG: |
                org.apache.kafka.common.security.plain.PlainLoginModule required \
                username="${KAFKA_USERNAME}" \
                password="${KAFKA_PASSWORD}" \
                user_${KAFKA_USERNAME}="${KAFKA_PASSWORD}";
            KAFKA_JMX_PORT: 9997
        ports:
            - ${KAFKA_BROKER_2_PORT}:9094/tcp
        healthcheck:
            test: /opt/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092
            interval: 60s
            timeout: 10s
            retries: 3
            start_period: 1s
            start_interval: 1s
        depends_on:
            kafka-1:
                condition: service_healthy
            kafka-2:
                condition: service_healthy
            kafka-3:
                condition: service_healthy

    kafka-6:
        image: docker.io/apache/kafka:4.0.0
        restart: unless-stopped
        container_name: kafka-6.${DOMAIN_NAME}
        hostname: kafka-6.${DOMAIN_NAME}
        environment:
            KAFKA_CLUSTER_ID: ${KAFKA_CLUSTER_ID}
            KAFKA_NODE_ID: 6
            KAFKA_PROCESS_ROLES: broker
            KAFKA_LISTENERS: INTERNAL://:9092,EXTERNAL://:9094
            KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka-6:9092,EXTERNAL://${KAFKA_EXTERNAL_IP}:${KAFKA_BROKER_3_PORT}
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
            KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
            KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
            KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka-1.${DOMAIN_NAME}:9093,2@kafka-2.${DOMAIN_NAME}:9093,3@kafka-3.${DOMAIN_NAME}:9093
            KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
            KAFKA_LISTENER_NAME_EXTERNAL_PLAIN_SASL_JAAS_CONFIG: |
                org.apache.kafka.common.security.plain.PlainLoginModule required \
                username="${KAFKA_USERNAME}" \
                password="${KAFKA_PASSWORD}" \
                user_${KAFKA_USERNAME}="${KAFKA_PASSWORD}";
            KAFKA_JMX_PORT: 9997
        ports:
            - ${KAFKA_BROKER_3_PORT}:9094/tcp
        healthcheck:
            test: /opt/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092
            interval: 60s
            timeout: 10s
            retries: 3
            start_period: 1s
            start_interval: 1s
        depends_on:
            kafka-1:
                condition: service_healthy
            kafka-2:
                condition: service_healthy
            kafka-3:
                condition: service_healthy

    kafka-7:
        image: docker.io/apache/kafka:4.0.0
        restart: unless-stopped
        container_name: kafka-7.${DOMAIN_NAME}
        hostname: kafka-7.${DOMAIN_NAME}
        environment:
            KAFKA_CLUSTER_ID: ${KAFKA_CLUSTER_ID}
            KAFKA_NODE_ID: 7
            KAFKA_PROCESS_ROLES: broker
            KAFKA_LISTENERS: INTERNAL://:9092,EXTERNAL://:9094
            KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka-7:9092,EXTERNAL://${KAFKA_EXTERNAL_IP}:${KAFKA_BROKER_4_PORT}
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
            KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
            KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
            KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka-1.${DOMAIN_NAME}:9093,2@kafka-2.${DOMAIN_NAME}:9093,3@kafka-3.${DOMAIN_NAME}:9093
            KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
            KAFKA_LISTENER_NAME_EXTERNAL_PLAIN_SASL_JAAS_CONFIG: |
                org.apache.kafka.common.security.plain.PlainLoginModule required \
                username="${KAFKA_USERNAME}" \
                password="${KAFKA_PASSWORD}" \
                user_${KAFKA_USERNAME}="${KAFKA_PASSWORD}";
            KAFKA_JMX_PORT: 9997
        ports:
            - ${KAFKA_BROKER_4_PORT}:9094/tcp
        healthcheck:
            test: /opt/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092
            interval: 60s
            timeout: 10s
            retries: 3
            start_period: 1s
            start_interval: 1s
        depends_on:
            kafka-1:
                condition: service_healthy
            kafka-2:
                condition: service_healthy
            kafka-3:
                condition: service_healthy

    # ----- Kafka UI -----
    kafka-ui:
        image: ghcr.io/kafbat/kafka-ui:v1.2.0
        restart: unless-stopped
        container_name: kafka-ui.${DOMAIN_NAME}
        hostname: kafka-ui.${DOMAIN_NAME}
        read_only: true
        environment:
            KAFKA_CLUSTERS_0_NAME: ${KAFKA_CLUSTER_NAME}
            KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka-4.${DOMAIN_NAME}:9092,kafka-5.${DOMAIN_NAME}:9092,kafka-6.${DOMAIN_NAME}:9092,kafka-7.${DOMAIN_NAME}:9092
            KAFKA_CLUSTERS_0_METRICS_PORT: 9997
            KAFKA_CLUSTERS_0_METRICS_TYPE: JMX
            DYNAMIC_CONFIG_ENABLED: true
        ports:
            - "25280:8080"
        depends_on:
            kafka-4:
                condition: service_healthy
            kafka-5:
                condition: service_healthy
            kafka-6:
                condition: service_healthy
            kafka-7:
                condition: service_healthy

parameters:

# environment
DOMAIN_NAME=hello.com

# kafka
# external_ip should be your machine IP
KAFKA_CLUSTER_ID=01916ec2-824f-77b4-a3c2-c1ccfd480449
KAFKA_CLUSTER_NAME=kafka-cluster
KAFKA_USERNAME=admin
KAFKA_PASSWORD=password
KAFKA_EXTERNAL_IP=127.0.0.1
KAFKA_BROKER_1_PORT=25111
KAFKA_BROKER_2_PORT=25121
KAFKA_BROKER_3_PORT=25131
KAFKA_BROKER_4_PORT=25141

Steps to reproduce

start docker command:
docker compose --env-file="$environment_file" -f "$docker_compose_file" up

Screenshots

No response

Logs

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    status/triageIssues pending maintainers triagetype/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions