Performance test improvements #384

This PR enhances performance testing capabilities by adding detailed metrics collection including eachBatch processing, memory usage tracking, timing measurements, and offset lag monitoring. The changes consolidate common consumer functionality and expand test coverage to include both eachMessage and eachBatch consumption patterns.

Key changes:

Extracted common consumer logic into a shared module supporting both eachMessage and eachBatch patterns
Added comprehensive memory tracking with RSS measurements during test execution
Enhanced metrics collection including offset lag, processing time, and batch statistics
Migrated performance test script from Bash to Node.js for better cross-platform compatibility

Reviewed Changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
examples/performance/performance-primitives-common.js	New shared module implementing consumer logic for both eachMessage and eachBatch patterns with enhanced metrics
examples/performance/performance-primitives.js	Refactored to use common consumer module with compatibility wrapper
examples/performance/performance-primitives-kafkajs.js	Updated to use common consumer module and added duration logging
examples/performance/performance-consolidated.js	Added memory tracking, expanded test coverage for both consumption patterns, and enhanced logging
ci/tests/run_perf_test.sh	Removed Bash-based performance test script
ci/tests/run_perf_test.js	New Node.js-based performance test script with enhanced metric extraction
.semaphore/semaphore.yml	Updated to use Node.js performance test script

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

examples/performance/performance-primitives-common.js Outdated Show resolved Hide resolved

examples/performance/performance-primitives-common.js Show resolved Hide resolved

ci/tests/run_perf_test.js Outdated

Comment on lines 47 to 46

const outputConfluentProducerConsumer = runCommand('MODE=confluent MESSAGE_COUNT=50000 node performance-consolidated.js --create-topics --consumer --producer');

console.log('Running KafkaJS Producer/Consumer test...');

const outputKjsProducerConsumer = runCommand('MODE=kafkajs MESSAGE_COUNT=50000 node performance-consolidated.js --create-topics --consumer --producer');

Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The message count has been reduced from 500000 to 50000 compared to the original bash script. This significant reduction (90%) may affect the reliability of performance measurements and should be documented or reconsidered.

Suggested change

const outputConfluentProducerConsumer = runCommand('MODE=confluent MESSAGE_COUNT=50000 node performance-consolidated.js --create-topics --consumer --producer');

console.log('Running KafkaJS Producer/Consumer test...');

const outputKjsProducerConsumer = runCommand('MODE=kafkajs MESSAGE_COUNT=50000 node performance-consolidated.js --create-topics --consumer --producer');

const outputConfluentProducerConsumer = runCommand('MODE=confluent MESSAGE_COUNT=500000 node performance-consolidated.js --create-topics --consumer --producer');

console.log('Running KafkaJS Producer/Consumer test...');

const outputKjsProducerConsumer = runCommand('MODE=kafkajs MESSAGE_COUNT=500000 node performance-consolidated.js --create-topics --consumer --producer');

Copilot uses AI. Check for mistakes.

Copy link

Contributor Author

@emasab emasab Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it makes the test faster with bigger messages sizes and doesn't seem to affect reliability of results.

@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch 2 times, most recently from 99bbb70 to 7cd3c3f Compare

October 8, 2025 15:11

@emasab emasab changed the title ~~(削除) Performance test improvements, measure eachBatch rate, time and lag, avg and max memory usage (削除ここまで)~~ (追記) Performance test improvements (追記ここまで)

Oct 8, 2025

@sonarqube-confluent

This comment has been minimized.

1 similar comment

@sonarqube-confluent

This comment has been minimized.

milindl

milindl approved these changes

Oct 9, 2025

View reviewed changes

Copy link

Contributor

@milindl milindl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the new features to the test (the rss) and eachBatch. The run on semaphore also looks fine to me, as expected in terms of consume throughput and rss.

milindl

milindl reviewed

Oct 9, 2025

View reviewed changes

Copy link

Contributor

@milindl milindl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor changes suggested

examples/performance/performance-primitives-kafkajs.js

ret = {

...ret,

ssl: parameters.securityProtocol.toLowerCase().includes('ssl'),

sasl: {

Copy link

Contributor

@milindl milindl Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of doing it here, at a later point, do

if (parameters.securityProtocol.toLowerCase().includes('sasl')) {
 ret['sasl'] = { ... };
}

So we can have just ssl. And above && condition can be changed also

Copy link

Contributor

@milindl milindl Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for the confluent performance primitives.

@sonarqube-confluent

This comment has been minimized.

4 similar comments

@sonarqube-confluent

This comment has been minimized.

@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch 14 times, most recently from f2d1c71 to 1e7236e Compare

October 15, 2025 13:14

emasab added 11 commits

October 15, 2025 18:44

@emasab


 Run performance tests with SASL_PLAINTEXT or SASL_SSL and plain authe...

ebef2af

...ntication

@emasab


 Fixes

45c4097

@emasab


 Improve KafkaJS producing, more reliable createTopics, configurable m...

b71e51b

...essage count and partition number

@emasab


 Log number of partitions

d95680b

@emasab


 Fix skipCtpTest

33bd5ec

@emasab


 Configurable partitions consumed concurrently

e332111

@emasab


 Consume rate in msg/s, average batch size, different messages dependi...

073e0a6

...ng on randomness

@emasab


 fixup

8ffd977

@emasab


 Average and max E2E latency

948d590

@emasab


 Log average and max size

ac0d674

@emasab


 Changes to run the test for a longer time

9a5db79

@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch 2 times, most recently from 6a53af2 to e999d4d Compare

October 16, 2025 08:52

@sonarqube-confluent

This comment has been minimized.

1 similar comment

@sonarqube-confluent

This comment has been minimized.

@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch from e999d4d to 86f9ebf Compare

October 16, 2025 09:08

@emasab


 Add broker lag monitoring

ed0fb03

@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch from 86f9ebf to ed0fb03 Compare

October 16, 2025 09:20

@sonarqube-confluent

This comment has been minimized.

@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch from 9e73424 to ed0fb03 Compare

October 16, 2025 11:38

@sonarqube-confluent

This comment has been minimized.

1 similar comment

@sonarqube-confluent

This comment has been minimized.

@emasab


 fixup

476a38e

@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch 2 times, most recently from 15928f1 to 4209743 Compare

October 17, 2025 10:29

@sonarqube-confluent

This comment has been minimized.

@emasab


 Use same producer and different E2E latencies

9f4461f

@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch from 4209743 to 9f4461f Compare

October 17, 2025 11:02

@sonarqube-confluent

Copy link

sonarqube-confluent bot commented Oct 17, 2025

Passed

Analysis Details

10 Issues

Bug 0 Bugs
Vulnerability 4 Vulnerabilities
Code Smell 6 Code Smells

Coverage and Duplications

Coverage No coverage information (48.20% Estimated after merge)
Duplications No duplication information (2.00% Estimated after merge)

Project ID: confluent-kafka-javascript

View in SonarQube

@emasab


 Auto commit on batch end

77556ce

Labels

None yet

2 participants

@emasab @milindl

Performance test improvements #384

Are you sure you want to change the base?

Performance test improvements #384

Conversation

@emasab emasab commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

@emasab emasab Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

@milindl milindl left a comment

Choose a reason for hiding this comment

Uh oh!

@milindl milindl left a comment

Choose a reason for hiding this comment

Uh oh!

@milindl milindl Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

@milindl milindl Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

sonarqube-confluent bot commented Oct 17, 2025

Analysis Details

10 Issues

Coverage and Duplications

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

@emasab emasab commented Oct 8, 2025 •

edited

Loading