-
Notifications
You must be signed in to change notification settings - Fork 20
[core] fix field detection for escaped % and { format styles - #75
Open
dylanpulver wants to merge 2 commits into
Open
[core] fix field detection for escaped % and { format styles #75dylanpulver wants to merge 2 commits into
dylanpulver wants to merge 2 commits into
Conversation
`parse()` read `%%(name)s` and `{{name}}` as fields even though both are
escaped literals, and read a `{` style conversion or format spec as part
of the field name, so `{levelname:>8}` produced a `"levelname:>8": null`
key and dropped `levelname`.
`%` now skips `%%` the same way the `$` style skips `$$` (nhairs#69), and `{`
uses `string.Formatter`, which is what `logging.StrFormatStyle.validate`
parses the same string with.
Assisted-by: claude-opus-5
A regex-only fix passes the plain `{levelname:>8}` case but reads
`{width}` out of `{levelname:>{width}}`, which stdlib accepts.
Assisted-by: claude-opus-5
nhairs
nhairs
requested changes
Sep 12, 2026
@nhairs
nhairs
left a comment
Owner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @dylanpulver, overall looks good, though have a number of comments for you
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [4.2.1.dev1](https://github.com/nhairs/python-json-logger/compare/v4.2.0...main) - unreleased
Owner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit:
Suggested change
## [4.2.1.dev1](https://github.com/nhairs/python-json-logger/compare/v4.2.0...main) - unreleased
## [4.2.1](https://github.com/nhairs/python-json-logger/compare/v4.2.0...v4.2.1) - UNRELEASED
Comment on lines
+316
to
+318
# str.format escapes literal braces as {{ and }}, and a replacement field may
# carry a conversion (!r) or a format spec (:>10) that is not part of its name.
# string.Formatter is what logging.StrFormatStyle.validate itself parses with.
Owner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we need these comments
# PercentStyle is parent class of StringTemplateStyle and StrFormatStyle
# so it must be checked last.
return STYLE_PERCENT_REGEX.findall(self._fmt)
# %% is an escaped literal percent, so %%(name)s is not a field.
Owner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we need this comment
Comment on lines
+186
to
+201
@pytest.mark.parametrize("class_", ALL_FORMATTERS)
def test_percentage_format_escaped_percent(
env: LoggingEnvironment, class_: type[BaseJsonFormatter]
):
# Note: %% is an escaped literal percent, so %%(notafield)s is not a field
env.set_formatter(class_("%(levelname)s %(message)s 100%% %%(notafield)s"))
msg = "testing logging format"
env.logger.info(msg)
log_json = env.load_json()
assert log_json["message"] == msg
assert log_json.keys() == {"levelname", "message"}
return
Owner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be merged with test_percentage_format
Comment on lines
68
to
+69
STYLE_STRING_FORMAT_REGEX = re.compile(r"\{(.+?)\}", re.IGNORECASE) # { style
STYLE_PERCENT_REGEX = re.compile(r"%\((.+?)\)", re.IGNORECASE) # % style
# Deprecated: no longer used by `parse`, which uses `string.Formatter` instead.
Owner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should remove this rather than marking it as deprecated.
- `{` style formats now use `string.Formatter` (as `logging.StrFormatStyle.validate` does) to find
fields, so escaped literal braces (`{{`/`}}`) are skipped and a conversion (`{message!r}`) or
format spec (`{levelname:>8}`) is no longer treated as part of the field name.
Owner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should add your own
Thanks @dylanpulver line
Comment on lines
+10
to
+14
- `%` style formats no longer treat the escaped literal `%%` as the start of a field, so
`"%%(notafield)s"` is correctly read as literal text.
- `{` style formats now use `string.Formatter` (as `logging.StrFormatStyle.validate` does) to find
fields, so escaped literal braces (`{{`/`}}`) are skipped and a conversion (`{message!r}`) or
format spec (`{levelname:>8}`) is no longer treated as part of the field name.
Owner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These should have a reference to the issue
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #74.
parse()reads escaped literals as fields:%%(b)sin a%format,{{b}}in a{one. #69 fixed this for$; these two were left.{also folds a conversion or format spec into the name, so"{levelname:>8} {message}"— an ordinary stdlib format thatStrFormatStyle.validate()accepts — logs{"levelname:>8": null, "message": "hello"}.Changes
%: skip%%, mirroring the$fix.{: usestring.Formatter, which is whatlogging.StrFormatStyle.validateparses the same string with. Covers{{/}},!r,:>8, and nested specs like{levelname:>{width}}.STYLE_STRING_FORMAT_REGEXis kept but unused — say the word and I'll drop it.Testing
pytest tests: 224 passed, 218 before. Two new tests beside the existing per-style ones. black/pylint/mypy/validate-pyproject clean.Oracle: the substitution engine itself (
fmt % tracking_dict,string.Formatter().parse,Template.get_identifiers). Over 37 formats$agreed 7/7 as a control;%+{diverged 13x before, 1 after — the remainder is"{}", whichvalidate()skips too.Reverting only the
core.pychange fails the 6 new cases and nothing else. A regex-only alternative passes{levelname:>8}but readswidthout of{levelname:>{width}}; hence that case in the test.One behaviour change: with
validate=Falsea malformed{format now raises at construction instead of silently yielding no fields.Prepared with AI assistance; I reviewed and tested it locally.