- 
  Notifications
 You must be signed in to change notification settings 
- Fork 934
Set up type hinting: add fixes in schema-registry module (mostly already typed) #2107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🎉 All Contributor License Agreements have been signed. Ready to merge. 
Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.
 
 
 This comment has been minimized.
 
 
 
 
 This comment has been minimized.
 
 
 This comment has been minimized.
 
 
 
 
 This comment has been minimized.
 
 
 This comment has been minimized.
 
 
 
 
 This comment has been minimized.
54fab2a to
 a5ff8aa  
 Compare
 
 There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed the optional part as we are already checking emptiness for key_url in https://github.com/confluentinc/confluent-kafka-python/blob/master/src/confluent_kafka/schema_registry/rules/encryption/hcvault/hcvault_driver.py#L53, and I think HcVaultKmsClient is supposed to be only created with a specified key_uri, according to the function doc
Not sure if this is considered a breaking change (if customers initializes HcVaultKmsClient directly in their code). @rayokota would love to hear your thoughts on this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similar change as the one in hcvault_client.py
 
 
 This comment has been minimized.
 
 
 
 
 This comment has been minimized.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In practice the schema_str field should never be empty, and even it is I think it makes sense to raise the error to fail early here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple questions on the PR. We are introducing a lot of ignore comments in places like avro that feel off but maybe not worth tackling in this pass anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is there a need for type ignoring here? Can we set the ref/referenced_schema type to avoid this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In practice, subject and version of SchemaReference never be None, but probably for historical reason (or tech debt) they have been typed as optional. I think updating the types will be a breaking change
Here, get_version() requires non-empty subject and version, so that's why I added the type ignore there. Alternatively we can do:
 if ref.subject is None or ref.version is None:
 # maybe log something
 continue
 referenced_schema = await schema_registry_client.get_version(ref.subject, ref.version, True)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add a comment that 'type' in the dict is for the schema type and not any language type hinting? Seeing it might be confusing without context right next to a type ignore call.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should spread this function args out per line if we're type commenting one. It read weird like this and I think is error prone to refactor bugs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This changed the logic here. Are we certain it's a correct / tested change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_execute_rules_with_phase() already requires SerializationContext and subject to build RuleContext (
... entrypoint init files
a5ff8aa to
 cdbd203  
 Compare
 
 There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
 
 
 This comment has been minimized.
 
 
 
 
 This comment has been minimized.
- 75.50% Coverage on New Code (is less than 80.00%)
Analysis Details
42 Issues
- Bug 0 Bugs
- Vulnerability 0 Vulnerabilities
- Code Smell 42 Code Smells
Coverage and Duplications
- Coverage 75.50% Coverage (67.00% Estimated after merge)
- Duplications No duplication information (5.00% Estimated after merge)
Project ID: confluent-kafka-python
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I refactored the name_strategy functions a bit: they have to follow the same function signature but SerializationContext is only required for some
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moving the indentation is intended to address the type issue of logical_cluster and identity_pool for building _AsyncOAuthClient: they must be non-empty, which we already check in line 280 and 284
This doesn't affect code logic: we check self.bearer_auth_credentials_source in {'OAUTHBEARER', 'STATIC_TOKEN'} for the outer block, and those if-else branches are for OAUTHBEARER and STATIC_TOKEN respectively
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
JsonSchema union type is either bool or dict. This was likely coped from avro.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 42 out of 42 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
 
 
 
 Copilot
AI
 
 
 
 Oct 24, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Corrected incomplete error message from 'after the at' to 'after the attempt to generate it'.
 
 
 
 Copilot
AI
 
 
 
 Oct 24, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The elif condition check has been replaced with else without verifying all possible values. This assumes only 'OAUTHBEARER' and 'STATIC_TOKEN' are valid values for bearer_auth_credentials_source. Consider adding an explicit elif condition for 'STATIC_TOKEN' to make the code more maintainable and guard against unexpected values.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's already the outer block if self.bearer_auth_credentials_source in {'OAUTHBEARER', 'STATIC_TOKEN'}:
 
 
 
 Copilot
AI
 
 
 
 Oct 24, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The elif condition check has been replaced with else without verifying all possible values. This assumes only 'OAUTHBEARER' and 'STATIC_TOKEN' are valid values for bearer_auth_credentials_source. Consider adding an explicit elif condition for 'STATIC_TOKEN' to make the code more maintainable and guard against unexpected values.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's already the outer block if self.bearer_auth_credentials_source in {'OAUTHBEARER', 'STATIC_TOKEN'}: (line 273)
 
 
 
 Copilot
AI
 
 
 
 Oct 24, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The indentation is incorrect on line 867. The 'query['subject'] = subject_name' statement should be on its own line with proper indentation.
 
 
 
 Copilot
AI
 
 
 
 Oct 24, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The indentation is incorrect on line 868. The 'query['subject'] = subject_name' statement should be on its own line with proper indentation.
 
 
 src/confluent_kafka/admin/_group.py
 
 Outdated
 
 
 
 
 
 
 Copilot
AI
 
 
 
 Oct 24, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using a mutable default argument (empty list) is dangerous as it will be shared across all instances. Use None as default and initialize inside the function instead.
 
 
 
 Copilot
AI
 
 
 
 Oct 24, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing space after comma between 'inline_tags' and 'field_transformer' arguments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 37 out of 38 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
 
 
 
 Copilot
AI
 
 
 
 Oct 30, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The logic for calculating new_version is confusing. When is_expired is False, new_version should be 1, but when is_expired is True, it should be dek.version + 1. The current conditional expression has this backwards - it increments when is_expired is True but sets to 1 when False. Consider: new_version = (dek.version + 1) if is_expired else 1
 
 
 
 Copilot
AI
 
 
 
 Oct 30, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Calling .get() on a boolean value will raise AttributeError. After checking isinstance(schema, bool), the function should return immediately before attempting to call schema.get(). The early return is missing on line 213.
 
 
 
 Copilot
AI
 
 
 
 Oct 30, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding a ValueError when ctx is None changes the API behavior. Previously, these functions accepted Optional[SerializationContext], and None would result in an AttributeError at line 94. The new explicit check is clearer, but this is a breaking change that could affect existing code that catches AttributeError. Consider documenting this as a breaking change.
 
 
 
 Copilot
AI
 
 
 
 Oct 30, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] Converting float to int with int() truncates the decimal. If cache_capacity is 100.9, it becomes 100. Consider using round() instead for more intuitive behavior, or document that fractional values are truncated.
 
 
 
 Copilot
AI
 
 
 
 Oct 30, 2025 
 
 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] This appears to be a pure formatting change (moving parameters to separate lines) unrelated to type hinting. Such formatting changes should typically be in a separate commit to keep type hinting changes focused.
Uh oh!
There was an error while loading. Please reload this page.
What
Follow-up PR after #2041: adding missing types + correcting existing types, according to mypy static checker, in schema-registry module
What's left are functions that might require refactoring and more thorough investigation to get the types right:
Checklist
References
JIRA: https://confluentinc.atlassian.net/browse/DGS-22076
Test & Review
Open questions / Follow-ups