try now

read excerpt

pro 24ドル.99 per month

• access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
• choose one free eBook per month to keep
• exclusive 50% discount on all purchases
• renews monthly, pause or cancel renewal anytime

---

lite 19ドル.99 per month

• access to all Manning books, including MEAPs!

---

team

5, 10 or 20 seats+ for your team - learn more

---

Put on your platform architect hat! You’re a member of the development team at Piper Data Concepts (PDC), and your client is looking to modernize its workflow. An existing benchmarked development environment, made up of Kafka, Python, and Postgres, is at your disposal. Now it’s time to start conceptualizing the new and improved workflow. You’ll use Kafka to create an event-driven data pipeline, review and understand business requirements, use Python Poetry to package the project, write Python code using the Faust library to communicate with Kafka, and store the consumed data in a PostgreSQL database.

This project is designed for learning purposes and is not a complete, production-ready application or solution.

add to reading list ▾

Please log in to add this book to your reading list.

Rob Koch is a Principal Data Engineer at Slalom Build, and one of the community leaders of DeafintheCloud.com. He helps drive cloud-native architecture, blogs about migrating to the cloud and use of Lambdas, and has a passion for data- and event-driven systems. Having earned five AWS certifications (Cloud Practitioner, Big Data Specialty, DevOps Engineer Associate, SysOps Administrator Associate, and Solution Architect Associate), Robert is actively involved in the development community in Denver, often speaking at Denver Dev Day and the AWS Denver Meetup. Robert’s goal is to help the community understand the advantages of migrating to the cloud, being cloud-native, and having "serverless" applications and databases.

Shane Smith-Sahnow is a software engineer at Netlify. He has spent time at Github and New Relic working on large scale Kafka applications, developing analytical data pipelines, and shipping large scale features.