-
Couldn't load subscription status.
- Fork 23
Description
Hi Team,
We are migrating to Javascript client from KafkaJS and using APIs under:
const { Kafka } = require("@confluentinc/kafka-javascript").KafkaJS;
We would like to understand what does this statement mean in migration documentation present here: reference to doc
The statement mentions:
eachBatch’s batch size never exceeds 1.
However, doesn't that mean eachBatch will behave just like eachMessage since size is limited to 1?
I tried to do a quick POC with this snippet:
await consumer.run({
eachBatch: async ({ batch, resolveOffset, heartbeat, isRunning, isStale }) => {
console.log(`[${instanceId}] Received batch with ${batch.messages.length} messages`);
console.log(`[${instanceId}] Batch info:`, {
topic: batch.topic,
partition: batch.partition,
firstOffset: batch.firstOffset(),
lastOffset: batch.lastOffset()
});
for (const message of batch.messages) {
if (!isRunning() || isStale()) break;
console.log(`[${instanceId}] Processing message:`, {
offset: message.offset,
value: message.value.toString()
});
await resolveOffset(message.offset);
//await heartbeat();
console.log(`[${instanceId}] Started`);
for (let i = 0; i < 10000000; ) {
// Simulate processing delay
i++;
}
console.log(`[${instanceId}] Done`);
}
}
});
and I could see in logs that batch size is coming as more than 1 frequently, proving that no such limit is not in place.
So, can you please clarify what this statement means: eachBatch’s batch size never exceeds 1. and if there are any plans to limit eachBatch's size to 1 since it will affect our migration.
Thanks in advance.