Admin Client
๐ This release includes breaking changes for any application code that has not
๐
migrated from older Simple-style classes to newer Kafka-style classes.
IncompatibleBrokerVersion when passing an api_version (ian28223 / PR #1953)ConnectionError (jeffwidman / PR #1816)๐ This release is focused on KafkaConsumer performance, Admin Client
๐ improvements, and Client concurrency. The KafkaConsumer iterator implementation
๐ has been greatly simplified so that it just wraps consumer.poll(). The prior
๐ implementation will remain available for a few more releases using the optional
KafkaConsumer config: legacy_iterator=True . This is expected to improve
consumer throughput substantially and help reduce heartbeat failures / group
rebalancing.
Major thanks to @carsonip @Baisang @iv-m @davidheitman @cardy31 @ulrikjohansson @iAnomaly @Wayde2014 @ossdev07 @commanderdishwasher @justecorruptio @melor @rustyrothwurt @sachiin @jacky15 and @rikonen for submitting PRs; thanks as well to everyone that
submitted bug reports and issues, and to @jeffwidman and @tvoinarovskyi for code
โ
reviews, comments, testing, debugging, and helping to maintain kafka-python!
consumer.poll() for KafkaConsumer iteration (@dpkp / PR #1902)partitions_for_topic a read-through cache (@Baisang / PR #1781,#1809)ssl_cafile is not provided (@iAnomaly / PR #1883)sasl_kerberos_domain_name config to KafkaAdminClient (@jeffwidman / PR #1852)security_protocol config documentation for KafkaAdminClient (@cardy31 / PR #1849)_send_request_to_node() in KafkaAdminClient (@davidheitman / PR #1807)KafkaConsumer tests to pytest (@jeffwidman / PR #1886)KAFKA_VERSION env var in tests (@jeffwidman / PR #1887)socket.SOCK_STREAM in test assertions (@iv-m / PR #1879)consumer.topics() and consumer.partitions_for_topic() (@Baisang / PR #1829)api_version_auto_timeout_ms (@jeffwidman / PR #1812)๐ This is a patch release primarily focused on bugs related to concurrency,
โ
SSL connections and testing, and SASL authentication.
Major thanks to @pt2pham , @isamaru , @braedon , @gingercookiemage , for submitting
PRs to help fix many of these issues. And major thanks to everyone that
submitted bug reports and issues. And thanks always to @jeffwidman and @tvoinarovskyi for code
โ
reviews, comments, testing, debugging, and helping to maintain this project!
protocol.send_bytes (isamaru / PR #1752)state_change_callback with lock (dpkp / PR #1775)client._conns in send() (dpkp / PR #1772)client.check_version (dpkp / PR #1771)maybe_refresh_metadata -- it is only called by poll() (dpkp / PR #1769)๐ This release is primarily focused on addressing lock contention
and other coordination issues between the KafkaConsumer and the
๐ background heartbeat thread that was introduced in the 1.4 release.
skip_double_compressed_messages (jeffwidman / PR #1677)validate_only/include_synonyms (jeffwidman #1645)kafka.common internally (jeffwidman #1509)pylint (jeffwidman #1611)Unittest to pytest (jeffwidman #1620)ConnectionError (jeffwidman #1492)BrokerConnection.connection_delay() to return milliseconds (dpkp #1414)Fetcher._fetchable_partitions to avoid mutation errors (dpkp #1400)_unpack (j2gg0s #1403)BrokerConnection.connect_blocking() to improve bootstrap to multi-address hostnames (dpkp #1411)BrokerConnection.close() if already disconnected (dpkp #1424)api_version against known versions (dpkp #1434)max_records in KafkaConsumer.poll (dpkp #1398)