Answer a question

Ive got a local kafka running using the following docker-compose.yml

version: '2'
services:
  zookeeper:
    image: "confluentinc/cp-zookeeper:5.0.1"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  kafka:
    image: "confluentinc/cp-enterprise-kafka:5.0.1"
    ports:
      - '9092:9092'
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 100

Trying to run a basic create topic using kafka-client 2.1.0 in Scala:

val props = new Properties()
props.setProperty(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092")

val adminClient: AdminClient = AdminClient.create(props)
val newTopic = new NewTopic("test", 1, 1.toShort)
val topicsF = adminClient.createTopics(List(newTopic).asJavaCollection)
val result = topicsF.all().get()

but after some time I get:

org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.

I can create a topic using the command line:

kafka-topics --create \
    --zookeeper localhost:2181 \
    --replication-factor 1 \
    --partitions 1 \
    --topic test
Created topic "test".

kafka AdminClient API Timed out waiting for node assignment describes a similar problem using Java but the comment suggests that a system restart fixed the issue which is not the case on my side.

Answers

If you're running Kafka in Docker (or similar) you need to configure the listeners correctly. This article describes it in detail.

Here's an example of a Docker Compose that you can use to access Kafka from your host machine.

Disclaimer: I wrote the article :)