-
-
Notifications
You must be signed in to change notification settings - Fork 19.1k
BUG: read_sql wraps plain SQL strings in sqlalchemy.text() to allow % modulo and LIKE patterns (#35484) #62578
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
SergioGarcia00
wants to merge
4
commits into
pandas-dev:main
from
SergioGarcia00:fix/read_sql_percent
Open
BUG: read_sql wraps plain SQL strings in sqlalchemy.text() to allow % modulo and LIKE patterns (#35484) #62578
SergioGarcia00
wants to merge
4
commits into
pandas-dev:main
from
SergioGarcia00:fix/read_sql_percent
+56
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@SergioGarcia00 Can you update the description linking to the relevant issue?
@SergioGarcia00 Can you update the description linking to the relevant issue?
Oh, missed that sry. #35484
I dont really understand the errors i got by the CI, is there something i did wrong?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
The issue is coming from #3548
What does this PR do?
Fixes a long-standing issue where pd.read_sql fails to handle the percent character (%) when using SQLAlchemy engines.
Since pandas ≥ 1.1, the IO layer sets no_parameters=True for SQL execution, which causes plain string queries containing % to be misinterpreted as parameter placeholders.
As a result:
Queries like SELECT 1 % 2 raised UndefinedFunction errors.
Queries using LIKE 'Jo%' failed to match rows correctly.
This PR introduces a small helper _sa_text_if_string that wraps plain SQL strings in sqlalchemy.text() before execution.
This ensures that % is treated correctly, while SQLAlchemy select() objects continue to work as before.
Summary of changes
pandas/io/sql.py
Added _sa_text_if_string(stmt) helper.
Applied it inside SQLDatabase.read_query() before self.execute().
pandas/tests/io/sql/test_percent_patterns.py
Added new tests for:
SELECT 5 % 2 (modulo operator)
LIKE 'John%' (pattern matching)
SQLAlchemy selectable using (literal(7) % literal(3))
Other
Restored .gitignore to match upstream so only relevant files are modified.
How was this tested?
All pre-commit checks (ruff, codespell, etc.) pass locally.
pytest -q pandas/tests/io/sql/test_percent_patterns.py -ra → 3 tests passed.
Tested both on in-memory SQLite and PostgreSQL (PANDAS_TEST_POSTGRES_URI).
Notes for reviewers
The change only affects plain string queries.
SQLAlchemy expressions (select(), text(), etc.) remain unaffected.
A short "Bug Fixes" note will be added to doc/source/whatsnew/v3.x.y.rst.