Skip to content

Navigation Menu

Sign in
Sign up
This repository was archived by the owner on Jun 8, 2026. It is now read-only.

feat: Add support for multiplexed sessions - #1381

Merged
rahul2393 merged 3 commits into
main from
support-mux-session
May 28, 2025
Merged

feat: Add support for multiplexed sessions #1381
rahul2393 merged 3 commits into
main from
support-mux-session

Conversation

@rahul2393

@rahul2393 rahul2393 commented May 23, 2025

Copy link
Copy Markdown
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

rahul2393 requested review from a team May 23, 2025 19:42
@product-auto-label product-auto-label Bot added size: l Pull request size is large. api: spanner Issues related to the googleapis/python-spanner API. labels May 23, 2025
@product-auto-label product-auto-label Bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels May 24, 2025
@product-auto-label product-auto-label Bot added size: l Pull request size is large. and removed size: xl Pull request size is extra large. labels May 24, 2025
Comment thread google/cloud/spanner_v1/database.py
Comment thread tests/system/test_observability_options.py
Comment on lines +9 to +34
runs-on: ubuntu-latest

services:
emulator:
image: gcr.io/cloud-spanner-emulator/emulator:latest
ports:
- 9010:9010
- 9020:9020

steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.8
- name: Install nox
run: python -m pip install nox
- name: Run system tests
run: nox -s system
env:
SPANNER_EMULATOR_HOST: localhost:9010
GOOGLE_CLOUD_PROJECT: emulator-test-project
GOOGLE_CLOUD_TESTS_CREATE_SPANNER_INSTANCE: true
GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS: true
GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS_PARTITIONED_OPS: true

Check warning

Code scanning / CodeQL

Workflow does not contain permissions

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {{contents: read}}
@product-auto-label product-auto-label Bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels May 27, 2025
rahul2393 force-pushed the support-mux-session branch 4 times, most recently from e83298f to aa0e2a5 Compare May 27, 2025 10:20

rahul2393 commented May 27, 2025
edited
Loading

Copy link
Copy Markdown
Contributor Author
graph TD
 A[Client Request] --> B{Multiplexed Sessions Enabled?}
 
 B -->|No| C[Regular Session Behavior]
 B -->|Yes| D[Multiplexed Session Behavior]
 
 %% Regular Session Path
 C --> C1[Read-Only Operations]
 C --> C2[Partitioned Operations]
 C --> C3[Read-Write Operations]
 
 C1 --> C1A[Get Session from Pool]
 C1A --> C1B[Create/Use Regular Session]
 C1B --> C1C[Execute Read Operation]
 C1C --> C1D[Return Session to Pool]
 
 C2 --> C2A[BatchSnapshot Creation]
 C2A --> C2B[Get Session from Pool<br/>TransactionType.PARTITIONED]
 C2B --> C2C[Create Regular Session]
 C2C --> C2D[Generate Partitions]
 C2D --> C2E[Each Partition Uses<br/>SAME Session]
 C2E --> C2F[Process All Partitions<br/>Sequentially]
 
 C3 --> C3A[Get Session from Pool]
 C3A --> C3B[Create/Use Regular Session]
 C3B --> C3C[Execute Transaction]
 C3C --> C3D[Return Session to Pool]
 
 %% Multiplexed Session Path
 D --> D1[Read-Only Operations]
 D --> D2[Partitioned Operations]
 D --> D3[Read-Write Operations]
 
 D1 --> D1A[Get Multiplexed Session]
 D1A --> D1B{Multiplexed Session Exists?}
 D1B -->|No| D1C[Create Multiplexed Session]
 D1B -->|Yes| D1D[Reuse Existing Session]
 D1C --> D1E[Start Maintenance Thread]
 D1E --> D1F[Execute Read Operation]
 D1D --> D1F
 D1F --> D1G[Session Remains Active<br/>No Return Needed]
 
 D2 --> D2A[BatchSnapshot Creation]
 D2A --> D2B[Get Multiplexed Session<br/>TransactionType.PARTITIONED]
 D2B --> D2C{Multiplexed Session Exists?}
 D2C -->|No| D2D[Create Multiplexed Session]
 D2C -->|Yes| D2E[Reuse Existing Session]
 D2D --> D2F[Generate Partitions]
 D2E --> D2F
 D2F --> D2G[Each Partition Uses<br/>SAME Multiplexed Session]
 D2G --> D2H[Process All Partitions<br/>Concurrently Possible]
 
 D3 --> D3A[Fallback to Regular Session]
 D3A --> D3B[Get Session from Pool]
 D3B --> D3C[Create/Use Regular Session]
 D3C --> D3D[Execute Transaction]
 D3D --> D3E[Return Session to Pool]
 
 %% Environment Variables
 E[Environment Variables] --> E1[GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS=true]
 E --> E2[GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS_PARTITIONED_OPS=true]
 E1 --> B
 E2 --> D2
 
 %% Key Differences
 F[Key Differences] --> F1[Session Reuse]
 F --> F2[Concurrency]
 F --> F3[Resource Usage]
 
 F1 --> F1A[Regular: New session per operation]
 F1 --> F1B[Multiplexed: Same session shared]
 
 F2 --> F2A[Regular: Sequential partition processing]
 F2 --> F2B[Multiplexed: Concurrent partition processing]
 
 F3 --> F3A[Regular: Multiple sessions = more resources]
 F3 --> F3B[Multiplexed: Single session = fewer resources]
 
 %% Styling
 classDef regularPath fill:#ffcccc,stroke:#ff0000,stroke-width:2px
 classDef multiplexedPath fill:#ccffcc,stroke:#00ff00,stroke-width:2px
 classDef envVar fill:#ffffcc,stroke:#ffaa00,stroke-width:2px
 classDef keyDiff fill:#ccccff,stroke:#0000ff,stroke-width:2px
 
 class C,C1,C2,C3,C1A,C1B,C1C,C1D,C2A,C2B,C2C,C2D,C2E,C2F,C3A,C3B,C3C,C3D regularPath
 class D,D1,D2,D3,D1A,D1B,D1C,D1D,D1E,D1F,D1G,D2A,D2B,D2C,D2D,D2E,D2F,D2G,D2H multiplexedPath
 class E,E1,E2 envVar
 class F,F1,F2,F3,F1A,F1B,F2A,F2B,F3A,F3B keyDiff
Loading

Comment thread tests/system/test_session_api.py Outdated
Comment on lines +3356 to +3340
rows1 = list(snap1.read(sd.TABLE, sd.COLUMNS, sd.ALL))
rows2 = list(snap2.read(sd.TABLE, sd.COLUMNS, sd.ALL))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests seem to be failing here because the snapshot's session has does not exist. This means that Session.create() has not been called. Not sure exactly why that wouldn't be the case, but tough to find out without being able to run locally.

@rahul2393 rahul2393 May 27, 2025
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue was some tests were calling session.delete which prematurely deletes the multiplex session and hence "Session Not Found" error, Using Emulator calling session.delete() does not actually delete it from Emulator so issue does not occur using Emulator

rahul2393 requested review from sakthivelmanii and removed request for olavloite May 27, 2025 18:15
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers
@currantw currantw currantw left review comments
@sakthivelmanii sakthivelmanii sakthivelmanii approved these changes
Reviewers whose approvals may not affect merge requirements

Labels

api: spanner Issues related to the googleapis/python-spanner API. size: xl Pull request size is extra large.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /