Access kafka on docker from host machine

Access kafka on docker from host machine

Problem

You can easily create kafka container using docker.

We are using docker compose to create kafka, storm container.

This setup is working fine but we cannot send any message from host machine to kafka running on docker container.

All ports are currently mapped to host machine.

I can ssh to docker container and send message but i was not able to send any message from my host.

It is very irritating as i cannot use application running in Intellij to send any message to kafka on docker.

Solution

When i use docker compose to create docker container i use following configuration

kafka:    
    image: docker image
    ports: - "2181:2181"    
           - "9092:9092"  
           - "9000:9000"
 environment:  
 KAFKA_CREATE_TOPICS: topic1
 ADVERTISED_HOST: kafka

Notice that we set ADVERTISER_HOST as kafka.

Now open your host file and put following entries(this depends on environment linux/windows/mac).

<<ip address of docker machine>> kafka

Now in your program use kafka as zookeeper host instead of ip address.