Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Comments

Fix pandas 3.0 compatibility#561

Open
ecomodeller wants to merge 7 commits intomain from
fix/pandas3-compatibility
Open

Fix pandas 3.0 compatibility #561
ecomodeller wants to merge 7 commits intomain from
fix/pandas3-compatibility

Conversation

@ecomodeller
Copy link
Member

@ecomodeller ecomodeller commented Jan 22, 2026
edited
Loading

Fix pandas 3.0 compatibility issues

Critical fixes:

  • Normalize datetime coordinates to datetime64[ns] at data loading to prevent xarray interpolation failures caused by pandas 3.0's default datetime resolution change
  • Replace np.issubdtype() with pd.api.types.is_numeric_dtype() for pandas 3.0 StringDtype compatibility
  • Fix peak_ratio metric to use .total_seconds() instead of hardcoded nanosecond division
  • Add pd.notna() checks before accessing .mid in pd.cut() operations
  • Use .get("kind") instead of ["kind"] to handle auxiliary variables safely

Additional improvements:

  • Update deprecated frequency strings to lowercase (H→h, S→s)
  • Add explicit compat and join parameters to xarray merge to avoid FutureWarnings
  • Improve peak_ratio test with deterministic synthetic data

Maintains backward compatibility with pandas 2.x while supporting pandas 3.0.

- Update deprecated frequency strings to lowercase (H→h, S→s)
- Add explicit compat and join parameters to xarray merge to avoid FutureWarnings
- Apply ruff formatting
- Add Git workflow guidelines to CLAUDE.md
Changes:
- src/modelskill/comparison/_comparison.py: Add explicit compat="no_conflicts" and join="outer" to merge()
- src/modelskill/model/dummy.py: Change freq="H" to freq="h" in docstring example
- tests/test_simple_compare.py: Change freq="4S" to freq="4s"
- CLAUDE.md: Add Git Workflow section with reminder to never commit directly to main
- Format code with ruff
All tests pass (589 passed, 5 skipped) with only expected UserWarnings remaining.
Pandas 3.0 changes default datetime precision from datetime64[ns] to
datetime64[us], which causes xarray.interp() to fail when source and
target datasets have mismatched precision. Additionally, pd.cut() NaN
handling and StringDtype compatibility changed.
Changes:
- Normalize datetime coordinates to datetime64[ns] for backward
 compatibility with pandas 2.x in _convert_to_dataset() and
 _parse_dataset()
- Fix pd.cut() NaN handling: check pd.notna(x) before accessing .mid
- Replace np.issubdtype() with pd.api.types.is_numeric_dtype() for
 pandas 3.0 StringDtype compatibility
- Fix peak_ratio metric: use .total_seconds() instead of hardcoded
 nanosecond division to work with any datetime precision
- Replace peak_ratio test with synthetic data test that has verifiable
 peaks (ratio=1.1)
- Improve code clarity: iterate over self.data instead of matched_data,
 use .get("kind") to handle auxiliary variables safely
- Remove unused imports in test files (ruff auto-fix)
@ecomodeller ecomodeller changed the title (削除) Fix pandas 3.0 compatibility warnings (削除ここまで) (追記) Fix pandas 3.0 compatibility (追記ここまで) Jan 22, 2026
self.raw_mod_data = {
str(key): PointModelResult(self.data[[str(key)]], name=str(key))
for key, value in matched_data.data_vars.items()
for key, value in self.data.data_vars.items()
Copy link
Member Author

@ecomodeller ecomodeller Feb 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the key change, using self.data already validated and fixed, instead of matched_data.

Reverted changes that were not related to Pandas 3 compatibility:
- CLAUDE.md git workflow docs
- _comparer_plotter.py formatting
- _comparison.py raw_mod_data refactoring and merge params
- Test file formatting and unused import cleanup
These changes are tracked in issues #568 and #569 for separate PRs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@stkistner stkistner Awaiting requested review from stkistner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

AltStyle によって変換されたページ (->オリジナル) /