Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Implement RdKafka getAssignment Method #1376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lucas-angeli-gimenes wants to merge 1 commit into php-enqueue:master
base: master
Choose a base branch
Loading
from lucas-angeli-gimenes:kafka-consumer-get-assignment

Conversation

Copy link

@lucas-angeli-gimenes lucas-angeli-gimenes commented Mar 7, 2025

The getAssignment method in RdKafka is useful for verifying the health of a Kafka consumer by checking if it has active partition assignments.

Benefits:

  1. Ensuring the Consumer is Active

    • A healthy consumer should have at least one assigned partition.
    • If getAssignment returns an empty list, it may indicate that the consumer is not properly connected or is not receiving partition assignments.
  2. Detecting Rebalance Issues

    • Kafka dynamically reassigns partitions when consumers join or leave a group.
    • A consumer that remains without assigned partitions for an extended period might be experiencing rebalance failures.
  3. Identifying Invalid States

    • If a consumer is running but has no assigned partitions, possible issues include:
      • The consumer failed to join the group.
      • Kafka has no partitions available for assignment.
      • There is a communication failure between the consumer and the broker.
  4. Health Check Integration for Auto-Scaling

    • This method can be used in readiness and liveness probes to ensure that only consumers with assigned partitions remain operational, preventing idle instances.

Example Usage:

$consumer = new RdKafka\KafkaConsumer($config);
$assignment = [];
$consumer->getAssignment($assignment);
if (empty($assignment)) {
 echo "❌ No partitions assigned! The consumer might not be connected to Kafka.";
} else {
 echo "✅ Consumer is healthy! Assigned partitions: " ;
}

By incorporating getAssignment into health checks, we can improve the resilience of our Kafka consumer infrastructure and detect issues proactively.

Copy link

@luizmanhani luizmanhani left a comment
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

medeirosinacio reacted with heart emoji
Copy link

Inglês brabo!

medeirosinacio reacted with hooray emoji

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
5 more reviewers

@wilcorrea wilcorrea wilcorrea approved these changes

@medeirosinacio medeirosinacio medeirosinacio approved these changes

@rafascosta rafascosta rafascosta approved these changes

@jonasntsilva jonasntsilva jonasntsilva approved these changes

@luizmanhani luizmanhani luizmanhani approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /