Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

HAProxy Configuration: network.loadbalancer.haproxy.idle.timeout#12586

Open
bradh352 wants to merge 2 commits intoapache:main from
bradh352:haproxy_timeout
Open

HAProxy Configuration: network.loadbalancer.haproxy.idle.timeout #12586
bradh352 wants to merge 2 commits intoapache:main from
bradh352:haproxy_timeout

Conversation

@bradh352
Copy link
Contributor

@bradh352 bradh352 commented Feb 4, 2026
edited
Loading

Description

This PR adds a new configuration parameter of network.loadbalancer.haproxy.idle.timeout which controls the HAProxy timeout server and timeout client configuration parameters under the defaults section.

Fixes #12574

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

  • Embedded unit tests to ensure generated configuration file matches.
  • Manual testing

How did you try to break this feature and the system with this change?

It has been observed that this will not update dynamically unless some other load balancer configuration is changed (or the VR is restarted). It is not clear if there is any infrastructure to 'trigger' a push to all existing load balancers to regenerate their configuration since this is global.

Copy link
Contributor

@blueorangutan package

Copy link

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

Copy link

codecov bot commented Feb 5, 2026
edited
Loading

Codecov Report

❌ Patch coverage is 64.70588% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.90%. Comparing base (a5b6bc3) to head (f5216af).

Files with missing lines Patch % Lines
...oud/network/lb/ElasticLoadBalancerManagerImpl.java 0.00% 2 Missing ⚠️
...a/com/cloud/network/router/CommandSetupHelper.java 0.00% 2 Missing ⚠️
...in/java/com/cloud/network/HAProxyConfigurator.java 83.33% 0 Missing and 1 partial ⚠️
...ork/router/VirtualNetworkApplianceManagerImpl.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@
## main #12586 +/- ##
=========================================
 Coverage 17.89% 17.90% 
- Complexity 16092 16094 +2 
=========================================
 Files 5936 5936 
 Lines 532734 532747 +13 
 Branches 65165 65167 +2 
=========================================
+ Hits 95347 95363 +16 
+ Misses 426711 426707 -4 
- Partials 10676 10677 +1 
Flag Coverage Δ
uitests 3.68% <ø> (ø)
unittests 19.00% <64.70%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new global configuration parameter network.loadbalancer.haproxy.idle.timeout to control HAProxy's timeout client and timeout server directives in the defaults section. The default value is 50000 milliseconds (matching the current hardcoded value), and setting it to 0 removes the timeout directives entirely for infinite timeout. This addresses issue #12574 where the aggressive 50-second timeout was problematic for long-running database connections through load balancers.

Changes:

  • Added NETWORK_LB_HAPROXY_IDLE_TIMEOUT configuration key with default value of 50000ms
  • Updated LoadBalancerConfigCommand to include idleTimeout parameter
  • Modified HAProxyConfigurator to generate timeout directives based on the configuration value
  • Added health check validation for the idle timeout configuration
  • Updated all LoadBalancerConfigCommand instantiations across VR, internal LB, and elastic LB implementations
  • Added comprehensive unit tests for timeout configuration generation

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
engine/api/src/main/java/org/apache/cloudstack/engine/orchestration/service/NetworkOrchestrationService.java Defines the new configuration key for HAProxy idle timeout
engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java Registers the new configuration key in the configurable keys array
core/src/main/java/com/cloud/agent/api/routing/LoadBalancerConfigCommand.java Adds idleTimeout field and updates constructor signature
core/src/main/java/com/cloud/network/HAProxyConfigurator.java Implements logic to set or remove timeout directives based on idleTimeout value
server/src/main/java/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java Passes idle timeout value to load balancing data for virtual routers
server/src/main/java/com/cloud/network/router/CommandSetupHelper.java Updates LoadBalancerConfigCommand instantiation with idle timeout value
plugins/network-elements/internal-loadbalancer/src/main/java/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java Updates LoadBalancerConfigCommand instantiation for internal load balancers
plugins/network-elements/elastic-loadbalancer/src/main/java/com/cloud/network/lb/ElasticLoadBalancerManagerImpl.java Updates LoadBalancerConfigCommand instantiation for elastic load balancers
systemvm/debian/root/health_checks/haproxy_check.py Adds health check validation for idle timeout configuration (contains bugs)
core/src/test/java/com/cloud/network/HAProxyConfiguratorTest.java Adds tests for timeout configuration with 0 and non-zero values
core/src/test/java/com/cloud/agent/resource/virtualnetwork/VirtualRoutingResourceTest.java Updates test fixtures with idle timeout parameter
core/src/test/java/com/cloud/agent/resource/virtualnetwork/ConfigHelperTest.java Updates test fixtures with idle timeout parameter

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +32 to +39
if "idletimeout" in haproxyData:
if "timeout client" not in haCfgSections["defaults"] or "timeout server" not in haCfgSections["defaults"]:
print("defaults timeout client or timeout server missing")

if haproxyData["idletimeout"] != haCfgSections["defaults"]["timeout client"][0].strip() or haproxyData["idletimeout"] != haCfgSections["defaults"]["timeout server"][0].strip():
print("defaults timeout client or timeout server mismatch occurred")
return False

Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The health check function has multiple critical bugs:

  1. The config file parser splits lines by space with maxsplit=1 (line 126), so "timeout client 1234" becomes key="timeout" with value="client 1234". However, this function tries to access keys "timeout client" and "timeout server" which don't exist in the parsed configuration.

  2. When the timeout fields are missing (line 33-34), the function prints an error but doesn't return False, allowing execution to continue to line 36 where it will crash trying to access non-existent keys.

  3. The check at line 36 cannot handle the idletimeout=0 case. When idletimeout is 0, HAProxyConfigurator sets these lines to empty strings (HAProxyConfigurator.java lines 644-645), which means they won't be present in the parsed config.

The correct approach would be to:

  • Check for the "timeout" key, not "timeout client"/"timeout server"
  • Extract and parse the timeout values from haCfgSections["defaults"]["timeout"] list
  • Handle the case where idletimeout is 0 or "0" by verifying the timeout entries are missing or empty
  • Return False immediately when timeout fields are missing (if idletimeout != 0)
Suggested change
if "idletimeout" in haproxyData:
if "timeout client" not in haCfgSections["defaults"] or "timeout server" not in haCfgSections["defaults"]:
print("defaults timeout client or timeout server missing")
if haproxyData["idletimeout"] != haCfgSections["defaults"]["timeout client"][0].strip() or haproxyData["idletimeout"] != haCfgSections["defaults"]["timeout server"][0].strip():
print("defaults timeout client or timeout server mismatch occurred")
return False
if "idletimeout" not in haproxyData:
return True
# Normalize idletimeout value to string for comparison
idle_value = str(haproxyData["idletimeout"]).strip()
# Safely get the defaults section and its timeout directives
defaults_section = haCfgSections.get("defaults", {})
timeout_lines = defaults_section.get("timeout", [])
# Extract client and server timeout values from the parsed "timeout" entries
timeout_values = {}
for tline in timeout_lines:
tline = tline.strip()
if not tline:
continue
parts = tline.split(None, 1)
if len(parts) < 2:
continue
kind, value = parts[0].strip(), parts[1].strip()
if kind in ("client", "server"):
timeout_values[kind] = value
# Special handling for idletimeout == 0: there should be no client/server timeouts configured
if idle_value == "0":
if "client" in timeout_values or "server" in timeout_values:
print("defaults timeout client or timeout server should be absent when idletimeout is 0")
return False
return True
# Non-zero idletimeout: both client and server timeouts must be present
if "client" not in timeout_values or "server" not in timeout_values:
print("defaults timeout client or timeout server missing")
return False
if idle_value != timeout_values["client"] or idle_value != timeout_values["server"]:
print("defaults timeout client or timeout server mismatch occurred")
return False

Copilot uses AI. Check for mistakes.
// .remove() is not allowed, only .set() operations are allowed as the list
// is a fixed size. So lets just mark the entry as blank.
dSection.set(9, "");
dSection.set(10, "");
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code doesn't handle negative values for idleTimeout. If a user somehow configures a negative value, it would fall through the if-else logic and retain the default timeout values from defaultsSection (50000). Consider adding validation to ensure idleTimeout is non-negative, or explicitly handle negative values in the else clause to make the behavior more predictable.

Suggested change
dSection.set(10, "");
dSection.set(10, "");
} else {
// Negative idleTimeout values are considered invalid; retain the
// default HAProxy timeout values from defaultsSection for predictability.
logger.warn("Negative idleTimeout ({}) configured; retaining default HAProxy timeouts.", lbCmd.idleTimeout);

Copilot uses AI. Check for mistakes.
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16705

Copy link
Contributor

Copy link

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@weizhouapache weizhouapache Awaiting requested review from weizhouapache

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

HAProxy load balancer client/server timeouts not appropriate for all use cases

AltStyle によって変換されたページ (->オリジナル) /