- simple python kafka app with setup mentioned below .
- producer produces contents of csv to a topic
- consumer consumes contents and pushes to a mongoDb through pyMongo client
- C:<kafka_2.13-2.5.0>
- cd C:\kafka_2.13-2.5.0
- bin\windows\zookeeper-server-start.bat config\zookeeper.properties
- bin\windows\kafka-server-start.bat config\server.properties
- Topic - > (communication channel) -> use of partitions allowed
- Producer - > push data - > Consumer registered picks it.
- bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --partitions 1 --replication-factor 1 --topic csvdata
- bin\windows\kafka-topics.bat --list --zookeeper localhost:2181
- bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic csvdata
- bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic csvdata
- bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic csvdata < mstatistics.csv