Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 4f7a936

Browse files
authored
[WIP] Retries mechanism improvements (#5103)
1 parent 73960e8 commit 4f7a936

File tree

6 files changed

+1130
-0
lines changed

6 files changed

+1130
-0
lines changed

‎docs/retry-mechanisms-enhancement.md

Lines changed: 237 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,237 @@
1+
# Retry Mechanisms Enhancement
2+
3+
This document describes the improvements made to CodeceptJS retry mechanisms to eliminate overlaps and provide better coordination.
4+
5+
## Problem Statement
6+
7+
CodeceptJS previously had multiple retry mechanisms that could overlap and conflict:
8+
9+
1. **Global Retry Configuration** - Feature and Scenario level retries
10+
2. **RetryFailedStep Plugin** - Individual step retries
11+
3. **Manual Step Retries** - `I.retry()` calls
12+
4. **Hook Retries** - Before/After hook retries
13+
5. **Helper Retries** - Helper-specific retry mechanisms
14+
15+
These mechanisms could result in:
16+
17+
- Exponential retry counts (e.g., 3 scenario retries ×ばつ 2 step retries = 6 total executions per step)
18+
- Conflicting configurations with no clear precedence
19+
- Confusing logging and unclear behavior
20+
- Difficult debugging when multiple retry levels were active
21+
22+
## Solution Overview
23+
24+
### 1. Enhanced Global Retry (`lib/listener/enhancedGlobalRetry.js`)
25+
26+
**New Features:**
27+
28+
- Priority-based retry coordination
29+
- Clear precedence system
30+
- Enhanced logging with mechanism identification
31+
- Backward compatibility with existing configurations
32+
33+
**Priority System:**
34+
35+
```javascript
36+
const RETRY_PRIORITIES = {
37+
MANUAL_STEP: 100, // I.retry() or step.retry() - highest priority
38+
STEP_PLUGIN: 50, // retryFailedStep plugin
39+
SCENARIO_CONFIG: 30, // Global scenario retry config
40+
FEATURE_CONFIG: 20, // Global feature retry config
41+
HOOK_CONFIG: 10, // Hook retry config - lowest priority
42+
}
43+
```
44+
45+
### 2. Enhanced RetryFailedStep Plugin (`lib/plugin/enhancedRetryFailedStep.js`)
46+
47+
**New Features:**
48+
49+
- Automatic coordination with scenario-level retries
50+
- Smart deferral when scenario retries are configured
51+
- Priority-aware retry registration
52+
- Improved logging and debugging information
53+
54+
**Coordination Logic:**
55+
56+
- When scenario retries are configured, step retries are automatically deferred to avoid excessive retry counts
57+
- Users can override this with `deferToScenarioRetries: false`
58+
- Clear logging explains coordination decisions
59+
60+
### 3. Retry Coordinator (`lib/retryCoordinator.js`)
61+
62+
**New Features:**
63+
64+
- Central coordination service for all retry mechanisms
65+
- Configuration validation with warnings for potential conflicts
66+
- Retry registration and priority management
67+
- Summary reporting for debugging
68+
69+
**Key Functions:**
70+
71+
- `validateConfig()` - Detects configuration conflicts and excessive retry counts
72+
- `registerRetry()` - Registers retry mechanisms with priority coordination
73+
- `getEffectiveRetryConfig()` - Returns the active retry configuration for a target
74+
- `generateRetrySummary()` - Provides debugging information about active retry mechanisms
75+
76+
## Migration Guide
77+
78+
### Immediate Benefits (No Changes Needed)
79+
80+
The enhanced retry mechanisms are **backward compatible**. Existing configurations will continue to work with these improvements:
81+
82+
- Better coordination between retry mechanisms
83+
- Enhanced logging for debugging
84+
- Automatic conflict detection and resolution
85+
86+
### Recommended Configuration Updates
87+
88+
#### 1. For Simple Cases - Use Scenario Retries Only
89+
90+
**Old Configuration (potentially conflicting):**
91+
92+
```javascript
93+
module.exports = {
94+
retry: 3, // scenario retries
95+
plugins: {
96+
retryFailedStep: {
97+
enabled: true,
98+
retries: 2, // step retries - could result in 3 * 3 = 9 executions
99+
},
100+
},
101+
}
102+
```
103+
104+
**Recommended Configuration:**
105+
106+
```javascript
107+
module.exports = {
108+
retry: 3, // scenario retries only
109+
plugins: {
110+
retryFailedStep: {
111+
enabled: false, // disable to avoid conflicts
112+
},
113+
},
114+
}
115+
```
116+
117+
#### 2. For Step-Level Control - Use Step Retries Only
118+
119+
**Recommended Configuration:**
120+
121+
```javascript
122+
module.exports = {
123+
plugins: {
124+
retryFailedStep: {
125+
enabled: true,
126+
retries: 2,
127+
ignoredSteps: ['amOnPage', 'wait*'], // customize as needed
128+
},
129+
},
130+
// No global retry configuration
131+
}
132+
```
133+
134+
#### 3. For Mixed Scenarios - Use Enhanced Coordination
135+
136+
```javascript
137+
module.exports = {
138+
retry: {
139+
Scenario: 2, // scenario retries for most tests
140+
},
141+
plugins: {
142+
retryFailedStep: {
143+
enabled: true,
144+
retries: 1,
145+
deferToScenarioRetries: true, // automatically coordinate (default)
146+
},
147+
},
148+
}
149+
```
150+
151+
### Testing Your Configuration
152+
153+
Use the new retry coordinator to validate your configuration:
154+
155+
```javascript
156+
const retryCoordinator = require('codeceptjs/lib/retryCoordinator')
157+
158+
// Validate your configuration
159+
const warnings = retryCoordinator.validateConfig(yourConfig)
160+
if (warnings.length > 0) {
161+
console.log('Retry configuration warnings:')
162+
warnings.forEach(warning => console.log(' -', warning))
163+
}
164+
```
165+
166+
## Enhanced Logging
167+
168+
The new retry mechanisms provide clearer logging:
169+
170+
```
171+
[Global Retry] Scenario retries: 3
172+
[Step Retry] Deferred to scenario retries (3 retries)
173+
[Retry Coordinator] Registered scenario retry (priority: 30)
174+
```
175+
176+
## Breaking Changes
177+
178+
**None.** All existing configurations continue to work.
179+
180+
## New Configuration Options
181+
182+
### Enhanced RetryFailedStep Plugin
183+
184+
```javascript
185+
plugins: {
186+
retryFailedStep: {
187+
enabled: true,
188+
retries: 2,
189+
deferToScenarioRetries: true, // NEW: automatically coordinate with scenario retries
190+
minTimeout: 1000,
191+
maxTimeout: 10000,
192+
factor: 1.5,
193+
ignoredSteps: ['wait*', 'amOnPage']
194+
}
195+
}
196+
```
197+
198+
### New Options:
199+
200+
- `deferToScenarioRetries` (boolean, default: true) - When true, step retries are disabled if scenario retries are configured
201+
202+
## Debugging Retry Issues
203+
204+
### 1. Check Configuration Validation
205+
206+
```javascript
207+
const retryCoordinator = require('codeceptjs/lib/retryCoordinator')
208+
const warnings = retryCoordinator.validateConfig(Config.get())
209+
console.log('Configuration warnings:', warnings)
210+
```
211+
212+
### 2. Monitor Enhanced Logging
213+
214+
Run tests with `--verbose` to see detailed retry coordination logs.
215+
216+
### 3. Generate Retry Summary
217+
218+
```javascript
219+
// In your test hooks
220+
const summary = retryCoordinator.generateRetrySummary()
221+
console.log('Retry mechanisms active:', summary)
222+
```
223+
224+
## Best Practices
225+
226+
1. **Choose One Primary Retry Strategy** - Either scenario-level OR step-level retries, not both
227+
2. **Use Configuration Validation** - Check for conflicts before running tests
228+
3. **Monitor Retry Logs** - Use enhanced logging to understand retry behavior
229+
4. **Test Retry Behavior** - Verify your retry configuration works as expected
230+
5. **Avoid Excessive Retries** - High retry counts often indicate test stability issues
231+
232+
## Future Enhancements
233+
234+
- Integration with retry coordinator for all retry mechanisms
235+
- Runtime retry strategy adjustment
236+
- Retry analytics and reporting
237+
- Advanced retry patterns (exponential backoff, conditional retries)

‎lib/listener/enhancedGlobalRetry.js

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
const event = require('../event')
2+
const output = require('../output')
3+
const Config = require('../config')
4+
const { isNotSet } = require('../utils')
5+
6+
const hooks = ['Before', 'After', 'BeforeSuite', 'AfterSuite']
7+
8+
/**
9+
* Priority levels for retry mechanisms (higher number = higher priority)
10+
* This ensures consistent behavior when multiple retry mechanisms are active
11+
*/
12+
const RETRY_PRIORITIES = {
13+
MANUAL_STEP: 100, // I.retry() or step.retry() - highest priority
14+
STEP_PLUGIN: 50, // retryFailedStep plugin
15+
SCENARIO_CONFIG: 30, // Global scenario retry config
16+
FEATURE_CONFIG: 20, // Global feature retry config
17+
HOOK_CONFIG: 10, // Hook retry config - lowest priority
18+
}
19+
20+
/**
21+
* Enhanced global retry mechanism that coordinates with other retry types
22+
*/
23+
module.exports = function () {
24+
event.dispatcher.on(event.suite.before, suite => {
25+
let retryConfig = Config.get('retry')
26+
if (!retryConfig) return
27+
28+
if (Number.isInteger(+retryConfig)) {
29+
// is number - apply as feature-level retry
30+
const retryNum = +retryConfig
31+
output.log(`[Global Retry] Feature retries: ${retryNum}`)
32+
33+
// Only set if not already set by higher priority mechanism
34+
if (isNotSet(suite.retries())) {
35+
suite.retries(retryNum)
36+
suite.opts.retryPriority = RETRY_PRIORITIES.FEATURE_CONFIG
37+
}
38+
return
39+
}
40+
41+
if (!Array.isArray(retryConfig)) {
42+
retryConfig = [retryConfig]
43+
}
44+
45+
for (const config of retryConfig) {
46+
if (config.grep) {
47+
if (!suite.title.includes(config.grep)) continue
48+
}
49+
50+
// Handle hook retries with priority awareness
51+
hooks
52+
.filter(hook => !!config[hook])
53+
.forEach(hook => {
54+
const retryKey = `retry${hook}`
55+
if (isNotSet(suite.opts[retryKey])) {
56+
suite.opts[retryKey] = config[hook]
57+
suite.opts[`${retryKey}Priority`] = RETRY_PRIORITIES.HOOK_CONFIG
58+
}
59+
})
60+
61+
// Handle feature-level retries
62+
if (config.Feature) {
63+
if (isNotSet(suite.retries()) || (suite.opts.retryPriority || 0) <= RETRY_PRIORITIES.FEATURE_CONFIG) {
64+
suite.retries(config.Feature)
65+
suite.opts.retryPriority = RETRY_PRIORITIES.FEATURE_CONFIG
66+
output.log(`[Global Retry] Feature retries: ${config.Feature}`)
67+
}
68+
}
69+
}
70+
})
71+
72+
event.dispatcher.on(event.test.before, test => {
73+
let retryConfig = Config.get('retry')
74+
if (!retryConfig) return
75+
76+
if (Number.isInteger(+retryConfig)) {
77+
// Only set if not already set by higher priority mechanism
78+
if (test.retries() === -1) {
79+
test.retries(retryConfig)
80+
test.opts.retryPriority = RETRY_PRIORITIES.SCENARIO_CONFIG
81+
output.log(`[Global Retry] Scenario retries: ${retryConfig}`)
82+
}
83+
return
84+
}
85+
86+
if (!Array.isArray(retryConfig)) {
87+
retryConfig = [retryConfig]
88+
}
89+
90+
retryConfig = retryConfig.filter(config => !!config.Scenario)
91+
92+
for (const config of retryConfig) {
93+
if (config.grep) {
94+
if (!test.fullTitle().includes(config.grep)) continue
95+
}
96+
97+
if (config.Scenario) {
98+
// Respect priority system
99+
if (test.retries() === -1 || (test.opts.retryPriority || 0) <= RETRY_PRIORITIES.SCENARIO_CONFIG) {
100+
test.retries(config.Scenario)
101+
test.opts.retryPriority = RETRY_PRIORITIES.SCENARIO_CONFIG
102+
output.log(`[Global Retry] Scenario retries: ${config.Scenario}`)
103+
}
104+
}
105+
}
106+
})
107+
}
108+
109+
// Export priority constants for use by other retry mechanisms
110+
module.exports.RETRY_PRIORITIES = RETRY_PRIORITIES

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /