-
-
Notifications
You must be signed in to change notification settings - Fork 748
feat: Add configurable sensitive data masking with custom patterns #5109
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
136 changes: 125 additions & 11 deletions
docs/secrets.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,36 +1,150 @@ | ||
| # Secrets | ||
|
|
||
| It is possible to **mask out sensitive data** when passing it to steps. This is important when filling password fields, or sending secure keys to API endpoint. | ||
| It is possible to **mask out sensitive data** when passing it to steps. This is important when filling password fields, or sending secure keys to API endpoint. CodeceptJS provides two approaches for masking sensitive data: | ||
|
|
||
| ## 1. Using the `secret()` Function | ||
|
|
||
| Wrap data in `secret` function to mask sensitive values in output and logs. | ||
|
|
||
| For basic string `secret` just wrap a value into a string: | ||
|
|
||
| ```js | ||
| I.fillField('password', secret('123456')); | ||
| I.fillField('password', secret('123456')) | ||
| ``` | ||
|
|
||
| When executed it will be printed like this: | ||
|
|
||
| ``` | ||
| I fill field "password" "*****" | ||
| ``` | ||
|
|
||
| **Other Examples** | ||
|
|
||
| ```js | ||
| I.fillField('password', secret('123456')); | ||
| I.append('password', secret('123456')); | ||
| I.type('password', secret('123456')); | ||
| I.fillField('password', secret('123456')) | ||
| I.append('password', secret('123456')) | ||
| I.type('password', secret('123456')) | ||
| ``` | ||
|
|
||
| For an object, which can be a payload to POST request, specify which fields should be masked: | ||
|
|
||
| ```js | ||
| I.sendPostRequest('/login', secret({ | ||
| name: 'davert', | ||
| password: '123456' | ||
| }, 'password')) | ||
| I.sendPostRequest( | ||
| '/login', | ||
| secret( | ||
| { | ||
| name: 'davert', | ||
| password: '123456', | ||
| }, | ||
| 'password', | ||
| ), | ||
| ) | ||
| ``` | ||
|
|
||
| The object created from `secret` is as Proxy to the object passed in. When printed password will be replaced with \*\*\*\*. | ||
|
|
||
| > ⚠️ Only direct properties of the object can be masked via `secret` | ||
|
|
||
| ## 2. Global Sensitive Data Masking | ||
|
|
||
| CodeceptJS can automatically mask sensitive data in all output (logs, steps, debug messages, errors) using configurable patterns. This feature uses the `maskSensitiveData` configuration option. | ||
|
|
||
| ### Basic Usage (Boolean) | ||
|
|
||
| Enable basic masking with predefined patterns: | ||
|
|
||
| ```js | ||
| // codecept.conf.js | ||
| exports.config = { | ||
| // ... other config | ||
| maskSensitiveData: true, | ||
| } | ||
| ``` | ||
|
|
||
| This will mask common sensitive data patterns like: | ||
|
|
||
| - Authorization headers | ||
| - API keys | ||
| - Passwords | ||
| - Tokens | ||
| - Client secrets | ||
|
|
||
| ### Advanced Usage (Custom Patterns) | ||
|
|
||
| Define your own masking patterns: | ||
|
|
||
| ```js | ||
| // codecept.conf.js | ||
| exports.config = { | ||
| // ... other config | ||
| maskSensitiveData: { | ||
| enabled: true, | ||
| patterns: [ | ||
| { | ||
| name: 'Email', | ||
| regex: /(\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b)/gi, | ||
| mask: '[MASKED_EMAIL]', | ||
| }, | ||
| { | ||
| name: 'Credit Card', | ||
| regex: /\b(?:\d{4}[- ]?){3}\d{4}\b/g, | ||
| mask: '[MASKED_CARD]', | ||
| }, | ||
| { | ||
| name: 'Phone Number', | ||
| regex: /(\+?1[-.\s]?)?\(?([0-9]{3})\)?[-.\s]?([0-9]{3})[-.\s]?([0-9]{4})/g, | ||
| mask: '[MASKED_PHONE]', | ||
| }, | ||
| { | ||
| name: 'SSN', | ||
| regex: /\b\d{3}-\d{2}-\d{4}\b/g, | ||
| mask: '[MASKED_SSN]', | ||
| }, | ||
| ], | ||
| }, | ||
| } | ||
| ``` | ||
|
|
||
| ### Pattern Configuration | ||
|
|
||
| Each custom pattern object should have: | ||
|
|
||
| - `name`: A descriptive name for the pattern | ||
| - `regex`: A JavaScript regular expression to match the sensitive data | ||
| - `mask`: The replacement string to show instead of the sensitive data | ||
|
|
||
| ### Examples | ||
|
|
||
| With the above configuration: | ||
|
|
||
| **Input:** | ||
|
|
||
| ``` | ||
| User email: john.doe@company.com | ||
| Credit card: 4111 1111 1111 1111 | ||
| Phone: +1-555-123-4567 | ||
| ``` | ||
|
|
||
| **Output:** | ||
|
|
||
| ``` | ||
| User email: [MASKED_EMAIL] | ||
| Credit card: [MASKED_CARD] | ||
| Phone: [MASKED_PHONE] | ||
| ``` | ||
|
|
||
| ### Where Masking Applies | ||
|
|
||
| Global sensitive data masking is applied to: | ||
|
|
||
| - Step descriptions and output | ||
| - Debug messages (`--debug` mode) | ||
| - Log messages (`--verbose` mode) | ||
| - Error messages | ||
| - Success messages | ||
|
|
||
| > ⚠️ Direct `console.log()` calls in helper functions are not masked. Use CodeceptJS output functions instead. | ||
|
|
||
| The object created from `secret` is as Proxy to the object passed in. When printed password will be replaced with ****. | ||
| ### Combining Both Approaches | ||
|
|
||
| > ⚠️ Only direct properties of the object can be masked via `secret` | ||
| You can use both `secret()` function and global masking together. The `secret()` function is applied first, then global patterns are applied to the remaining output. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
53 changes: 53 additions & 0 deletions
lib/utils/mask_data.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| const { maskSensitiveData } = require('invisi-data') | ||
|
|
||
| /** | ||
| * Mask sensitive data utility for CodeceptJS | ||
| * Supports both boolean and object configuration formats | ||
| * | ||
| * @param {string} input - The string to mask | ||
| * @param {boolean|object} config - Masking configuration | ||
| * @returns {string} - Masked string | ||
| */ | ||
| function maskData(input, config) { | ||
| if (!config) { | ||
| return input | ||
| } | ||
|
|
||
| // Handle boolean config (backward compatibility) | ||
| if (typeof config === 'boolean' && config === true) { | ||
| return maskSensitiveData(input) | ||
| } | ||
|
|
||
| // Handle object config with custom patterns | ||
| if (typeof config === 'object' && config.enabled === true) { | ||
| const customPatterns = config.patterns || [] | ||
| return maskSensitiveData(input, customPatterns) | ||
| } | ||
|
|
||
| return input | ||
| } | ||
|
|
||
| /** | ||
| * Check if masking is enabled based on global configuration | ||
| * | ||
| * @returns {boolean|object} - Current masking configuration | ||
| */ | ||
| function getMaskConfig() { | ||
| return global.maskSensitiveData || false | ||
| } | ||
|
|
||
| /** | ||
| * Check if data should be masked | ||
| * | ||
| * @returns {boolean} - True if masking is enabled | ||
| */ | ||
| function shouldMaskData() { | ||
| const config = getMaskConfig() | ||
| return config === true || (typeof config === 'object' && config.enabled === true) | ||
| } | ||
|
|
||
| module.exports = { | ||
| maskData, | ||
| getMaskConfig, | ||
| shouldMaskData, | ||
| } |
20 changes: 20 additions & 0 deletions
test/data/sandbox/codecept.bdd.boolean-masking.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| exports.config = { | ||
| tests: './*_no_test.js', | ||
| timeout: 10000, | ||
| output: './output', | ||
| helpers: { | ||
| BDD: { | ||
| require: './support/bdd_helper.js', | ||
| }, | ||
| }, | ||
| // Traditional boolean masking configuration | ||
| maskSensitiveData: true, | ||
| gherkin: { | ||
| features: './features/secret.feature', | ||
| steps: ['./features/step_definitions/my_steps.js', './features/step_definitions/my_other_steps.js'], | ||
| }, | ||
| include: {}, | ||
| bootstrap: false, | ||
| mocha: {}, | ||
| name: 'sandbox-boolean-masking', | ||
| } |
39 changes: 39 additions & 0 deletions
test/data/sandbox/codecept.bdd.masking.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| exports.config = { | ||
| tests: './*_no_test.js', | ||
| timeout: 10000, | ||
| output: './output', | ||
| helpers: { | ||
| BDD: { | ||
| require: './support/bdd_helper.js', | ||
| }, | ||
| }, | ||
| // New masking configuration with custom patterns | ||
| maskSensitiveData: { | ||
| enabled: true, | ||
| patterns: [ | ||
| { | ||
| name: 'Email', | ||
| regex: /(\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b)/gi, | ||
| mask: '[MASKED_EMAIL]', | ||
| }, | ||
| { | ||
| name: 'Credit Card', | ||
| regex: /\b(?:\d{4}[- ]?){3}\d{4}\b/g, | ||
| mask: '[MASKED_CARD]', | ||
| }, | ||
| { | ||
| name: 'Phone', | ||
| regex: /(\+?1[-.\s]?)?\(?([0-9]{3})\)?[-.\s]?([0-9]{3})[-.\s]?([0-9]{4})/g, | ||
| mask: '[MASKED_PHONE]', | ||
| }, | ||
| ], | ||
| }, | ||
| gherkin: { | ||
| features: './features/masking.feature', | ||
| steps: ['./features/step_definitions/my_steps.js', './features/step_definitions/my_other_steps.js'], | ||
| }, | ||
| include: {}, | ||
| bootstrap: false, | ||
| mocha: {}, | ||
| name: 'sandbox-masking', | ||
| } |
8 changes: 8 additions & 0 deletions
test/data/sandbox/features/masking.feature
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| Feature: Custom Data Masking | ||
|
|
||
| Scenario: mask custom sensitive data in output | ||
| Given I have user email "john.doe@example.com" | ||
| And I have credit card "4111 1111 1111 1111" | ||
| And I have phone number "+1-555-123-4567" | ||
| When I process user data | ||
| Then I should see masked output |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.