- 
  Notifications
 
You must be signed in to change notification settings  - Fork 312
 
Do not call default factories taking the data argument if a validation error already occurred #1623
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same needs to be done for DCs/TDs
CodSpeed Performance Report
Merging #1623 will not alter performance
Comparing default-factory-error (3ef7086) with main (70bd6f9)
Summary
✅ 163 untouched
...n error already occurred
a47bed9 to
 acce26b  
 Compare
 
  
 
 src/validators/with_default.rs
 
 Outdated
 
 
 There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a bit annoying because I don't have access to any input inside the default_value method. @davidhewitt any idea how I could workaround this in a clean way? I could wrap up the return type in an enum of either Option<PyObject> or a new singleton sentinel value, but the rtype is already complex enough 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, the current behavior is a bit weird:
class Model(BaseModel): a: int b: int Model(b="fails") """ pydantic_core._pydantic_core.ValidationError: 2 validation errors for Model a Field required [type=missing, input_value={'b': 'fails'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing b Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='fails', input_type=str] For further information visit https://errors.pydantic.dev/2.11/v/int_parsing """
For b, a value was provided and is used as input_value. For a, no value is provided and pydantic-core uses the whole input dict as input_value (on L214, input is the provided dict):
pydantic-core/src/validators/model_fields.rs
Lines 205 to 217 in 164b9ff
So perhaps we could just use PydanticUndefined as an input value here, and then in the with_default validator, I can return:
Err(ValError::new(ErrorTypeDefaults::DefaultFactoryNotCalled, &self.undefined))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm interesting, I wonder if the error for a should not include input_value or input_type at all? Or maybe they should be PydanticUndefined 🤔
maybe we could special-case PydanticUndefined in errors, so it would just show
Field required [type=missing]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll see if this can be done without too much special casing, I agree it would be better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to track this on the state? Should we adjust the algorithm to just never call .default_value if there's any errors?
(We could go further and rearrange things a bit so that all default values only get filled in after the whole input is processed, there is some overlap with #1620 🤔)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to track this on the state? Should we adjust the algorithm to just never call .default_value if there's any errors?
It's only relevant for default factories taking the data argument, so if you just have a "static" default it shouldn't matter. And because the logic to call the default value is nested under the current field validator, I can't really "influence" the logic from the model field.
This state approach doesn't look right, but I couldn't find a better way to do so 🤔
(We could go further and rearrange things a bit so that all default values only get filled in after the whole input is processed, there is some overlap with #1620 🤔)
Would this allow using other field values independently from the field order?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I see. I missed the default factories guarantee field order; this makes a lot of sense.
Because of the existence of PydanticUndefined as a way to ask for defaults, I think my original proposal actually doesn't make sense. 🤔
... but I do worry about when this flag is reset. Currently once you set it it's true for the rest of the validation. This seems problematic in unions, for example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed this is one of the reasons it doesn't look right; however, for unions I think it should be alright, as there's a boundary in the validation error.
The error we are handling here is the validation error of the field. If this field is defined like:
class Model(BaseModel): a: int | str
And a validation error happens on int, the field validator will try validating against str. At this point, we don't know what happened, and we will only get a val error if validation against str failed as well.
Edit: Actually it might be wrong if we have:
class Model1(BaseModel): a: int class Model2(BaseModel): ... class Model(BaseModel): m: Model1 | Model2
if validation fails for Model1.a, it will mutate the state (if it is the same state than when validating Model).
Now the question is: are unions the only concern? If so, we can make sure we somehow reset this state value on union validation, but there might be other concerns apart from unions :/
 
 
 src/validators/with_default.rs
 
 Outdated
 
 
 There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm interesting, I wonder if the error for a should not include input_value or input_type at all? Or maybe they should be PydanticUndefined 🤔
maybe we could special-case PydanticUndefined in errors, so it would just show
Field required [type=missing]
?
...Validation for a referenced field fails. This prevents a 'KeyNotFound' error from occuring. Fix is WIP by pydantic but not implemented yet. pydantic/pydantic-core#1623
...Validation for a referenced field fails. This prevents a 'KeyNotFound' error from occuring. Fix is WIP by pydantic but not implemented yet. pydantic/pydantic-core#1623
...Validation for a referenced field fails. This prevents a 'KeyNotFound' error from occuring. Fix is WIP by pydantic but not implemented yet. pydantic/pydantic-core#1623
fffc1b8 to
 f481aa6  
 Compare
 
 ...Validation for a referenced field fails. This prevents a 'KeyNotFound' error from occuring. Fix is WIP by pydantic but not implemented yet. pydantic/pydantic-core#1623
...n error already occurred (pydantic/pydantic-core#1623) Co-authored-by: David Hewitt <mail@davidhewitt.dev> Original-commit-hash: e87ba01
...n error already occurred (pydantic/pydantic-core#1623) Co-authored-by: David Hewitt <mail@davidhewitt.dev> Original-commit-link: pydantic/pydantic-core@e87ba01
Change Summary
Fixes pydantic/pydantic#11358.
@pydantic/oss, I'm not too satisfied with this approach, but I think it is reasonable? I just realized the same can also happen with validators:
But the default factory case strikes me as a much more common use case:
Related issue number
Checklist
pydantic-core(except for expected changes)