-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Replace asyncio to anyio
#3012
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace asyncio to anyio
#3012
Conversation
Docs Preview
963ac22
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@agronholm
agronholm
Oct 17, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit confused about this change. What's the story?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Honestly, I just remembered that I did something like this in Starlette and it solved a bug (maybe because it's only created on the async context) - I did the same here, and it solved a bug with Temporal.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What AnyIO version was this? Some time ago I refactored Lock to allow it to be instantiated outside of an async event loop, so it binds to the first one it's used in. Could the lack of that feature have been the cause?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The lock is on anyio 4.8.0.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we had a temporal script that was reproducing the bug without this, I'll see if I can find it again. Either way, if we convert to a property, we need to reuse the same value, so I'm not sure what the point would be of having it be a property and not an attribute. If the point is just to delay initialization I don't love the typing consequences (even though it's a private attribute). Either way I think the thing to do is try to reproduce the error without it first and then we can investigate other attempts to resolve the issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This script reproduces the issue if you move the cached_property back to an instance attribute created during initialization. Diff relative to the current state of this branch:
diff --git a/pydantic_ai_slim/pydantic_ai/agent/__init__.py b/pydantic_ai_slim/pydantic_ai/agent/__init__.py index 21fdb935..0c752e89 100644 --- a/pydantic_ai_slim/pydantic_ai/agent/__init__.py +++ b/pydantic_ai_slim/pydantic_ai/agent/__init__.py @@ -154,13 +154,14 @@ class Agent(AbstractAgent[AgentDepsT, OutputDataT]): _event_stream_handler: EventStreamHandler[AgentDepsT] | None = dataclasses.field(repr=False) + _enter_lock: anyio.Lock | None = dataclasses.field(repr=False, default_factory=anyio.Lock, init=False) _entered_count: int = dataclasses.field(repr=False) _exit_stack: AsyncExitStack | None = dataclasses.field(repr=False) - @functools.cached_property - def _enter_lock(self) -> anyio.Lock: - # We use a cached_property for this because it seems to work better with temporal... - return anyio.Lock() + # @functools.cached_property + # def _enter_lock(self) -> anyio.Lock: + # # We use a cached_property for this because it seems to work better with temporal... + # return anyio.Lock() @overload def __init__(
Script to repro:
from __future__ import annotations import asyncio from typing import Literal from temporalio import workflow from temporalio.client import Client from temporalio.testing import WorkflowEnvironment from temporalio.worker import Worker from pydantic_ai import Agent, RunContext from pydantic_ai.durable_exec.temporal import AgentPlugin, PydanticAIPlugin, TemporalAgent from pydantic_ai.exceptions import ApprovalRequired, CallDeferred from pydantic_ai.messages import ( ModelMessage, ModelRequest, ) from pydantic_ai.run import AgentRunResult from pydantic_ai.tools import DeferredToolRequests, DeferredToolResults TEMPORAL_PORT = 7233 TASK_QUEUE = 'pydantic-ai-agent-task-queue' hitl_agent = Agent( 'openai:gpt-5-mini', name='hitl_agent', output_type=[str, DeferredToolRequests], instructions='Just call tools without asking for confirmation.', ) @hitl_agent.tool def create_file(ctx: RunContext[None], path: str) -> None: raise CallDeferred @hitl_agent.tool def delete_file(ctx: RunContext[None], path: str) -> bool: if not ctx.tool_call_approved: raise ApprovalRequired return True hitl_temporal_agent = TemporalAgent(hitl_agent) @workflow.defn class HitlAgentWorkflow: def __init__(self): self._status: Literal['running', 'waiting_for_results', 'done'] = 'running' self._deferred_tool_requests: DeferredToolRequests | None = None self._deferred_tool_results: DeferredToolResults | None = None @workflow.run async def run(self, prompt: str) -> AgentRunResult[str | DeferredToolRequests]: messages: list[ModelMessage] = [ModelRequest.user_text_prompt(prompt)] while True: result = await hitl_temporal_agent.run( message_history=messages, deferred_tool_results=self._deferred_tool_results ) messages = result.all_messages() if isinstance(result.output, DeferredToolRequests): self._deferred_tool_requests = result.output self._deferred_tool_results = None self._status = 'waiting_for_results' await workflow.wait_condition(lambda: self._deferred_tool_results is not None) self._status = 'running' else: self._status = 'done' return result @workflow.query def get_status(self) -> Literal['running', 'waiting_for_results', 'done']: return self._status @workflow.query def get_deferred_tool_requests(self) -> DeferredToolRequests | None: return self._deferred_tool_requests @workflow.signal def set_deferred_tool_results(self, results: DeferredToolResults) -> None: self._status = 'running' self._deferred_tool_requests = None self._deferred_tool_results = results async def test_temporal_agent_with_hitl_tool(): async with await WorkflowEnvironment.start_local(port=TEMPORAL_PORT, ui=True) as env: # pyright: ignore[reportUnknownMemberType] client = await Client.connect( f'localhost:{TEMPORAL_PORT}', plugins=[PydanticAIPlugin()], ) async with Worker( client, task_queue=TASK_QUEUE, workflows=[HitlAgentWorkflow], plugins=[AgentPlugin(hitl_temporal_agent)], ): workflow = await client.start_workflow( # pyright: ignore[reportUnknownMemberType] HitlAgentWorkflow.run, args=['Delete the file `.env` and create `test.txt`'], id=HitlAgentWorkflow.__name__, task_queue=TASK_QUEUE, ) while True: import asyncio await asyncio.sleep(1) status = await workflow.query(HitlAgentWorkflow.get_status) # pyright: ignore[reportUnknownMemberType] if status == 'done': break elif status == 'waiting_for_results': # pragma: no branch deferred_tool_requests = await workflow.query(HitlAgentWorkflow.get_deferred_tool_requests) # pyright: ignore[reportUnknownMemberType] assert deferred_tool_requests is not None results = DeferredToolResults() # Approve all calls for tool_call in deferred_tool_requests.approvals: results.approvals[tool_call.tool_call_id] = True for tool_call in deferred_tool_requests.calls: results.calls[tool_call.tool_call_id] = 'Success' await workflow.signal(HitlAgentWorkflow.set_deferred_tool_results, results) result = await workflow.result() print(result.output) print(result.all_messages()) if __name__ == '__main__': asyncio.run(test_temporal_agent_with_hitl_tool())
The exception I get running the above:
/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/bin/python /Users/davidmontague/Library/Application Support/JetBrains/PyCharm2025.2/scratches/scratch_20.py CLI 1.4.1 (Server 1.28.0, UI 2.39.0) Server: localhost:7233 UI: http://localhost:8233 Metrics: http://localhost:51100/metrics __mro_entries__ on threading.local restricted Failed activation on workflow HitlAgentWorkflow with ID HitlAgentWorkflow and run ID 019a01e9-8e2e-7e77-9c37-06f7cbd205eb Traceback (most recent call last): File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 162, in get_async_backend return loaded_backends[asynclib_name] ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ KeyError: 'asyncio' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 418, in activate self._run_once(check_conditions=index == 1 or index == 2) File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2176, in _run_once raise self._current_activation_error File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2194, in _run_top_level_workflow_function await coro File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 986, in run_workflow result = await self._inbound.execute_workflow(input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2578, in execute_workflow return await input.run_fn(*args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Library/Application Support/JetBrains/PyCharm2025.2/scratches/scratch_20.py", line 58, in run result = await hitl_temporal_agent.run( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_agent.py", line 348, in run return await super().run( ^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/abstract.py", line 206, in run async with self.iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_agent.py", line 727, in iter async with super().iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/wrapper.py", line 194, in iter async with self.wrapped.iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/__init__.py", line 662, in iter async with toolset: File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/toolsets/combined.py", line 52, in __aenter__ async with self._enter_lock: ^^^^^^^^^^^^^^^^ File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/functools.py", line 995, in __get__ val = self.func(instance) ^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/toolsets/combined.py", line 41, in _enter_lock return anyio.Lock() ^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_synchronization.py", line 151, in __new__ return get_async_backend().create_lock(fast_acquire=fast_acquire) ^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 164, in get_async_backend module = import_module(f"anyio._backends._{asynclib_name}") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/__init__.py", line 3, in <module> from ._core._eventloop import current_time as current_time File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 497, in __call__ return self.current(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 248, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1470, in __import__ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 11, in <module> import sniffio File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 497, in __call__ return self.current(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 248, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1466, in __import__ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/sniffio/__init__.py", line 12, in <module> from ._impl import ( File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 497, in __call__ return self.current(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 248, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1470, in __import__ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/sniffio/_impl.py", line 11, in <module> class _ThreadLocal(threading.local): File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_restrictions.py", line 1016, in __getattribute__ state.assert_child_not_restricted(__name) File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_restrictions.py", line 852, in assert_child_not_restricted raise RestrictedWorkflowAccessError( temporalio.worker.workflow_sandbox._restrictions.RestrictedWorkflowAccessError: Cannot access threading.local.__mro_entries__ from inside a workflow. If this is code from a module not used in a workflow or known to only be used deterministically from a workflow, mark the import as pass through. __mro_entries__ on threading.local restricted Failed activation on workflow HitlAgentWorkflow with ID HitlAgentWorkflow and run ID 019a01e9-8e2e-7e77-9c37-06f7cbd205eb Traceback (most recent call last): File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 162, in get_async_backend return loaded_backends[asynclib_name] ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ KeyError: 'asyncio' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 418, in activate self._run_once(check_conditions=index == 1 or index == 2) File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2176, in _run_once raise self._current_activation_error File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2194, in _run_top_level_workflow_function await coro File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 986, in run_workflow result = await self._inbound.execute_workflow(input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2578, in execute_workflow return await input.run_fn(*args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Library/Application Support/JetBrains/PyCharm2025.2/scratches/scratch_20.py", line 58, in run result = await hitl_temporal_agent.run( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_agent.py", line 348, in run return await super().run( ^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/abstract.py", line 206, in run async with self.iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_agent.py", line 727, in iter async with super().iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/wrapper.py", line 194, in iter async with self.wrapped.iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/__init__.py", line 662, in iter async with toolset: File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/toolsets/combined.py", line 52, in __aenter__ async with self._enter_lock: ^^^^^^^^^^^^^^^^ File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/functools.py", line 995, in __get__ val = self.func(instance) ^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/toolsets/combined.py", line 41, in _enter_lock return anyio.Lock() ^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_synchronization.py", line 151, in __new__ return get_async_backend().create_lock(fast_acquire=fast_acquire) ^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 164, in get_async_backend module = import_module(f"anyio._backends._{asynclib_name}") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/__init__.py", line 3, in <module> from ._core._eventloop import current_time as current_time File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 497, in __call__ return self.current(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 248, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1470, in __import__ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 11, in <module> import sniffio File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 497, in __call__ return self.current(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 248, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1466, in __import__ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/sniffio/__init__.py", line 12, in <module> 2025-10-20T13:57:54.888574Z WARN temporal_sdk_core::worker::workflow: Failing workflow task run_id=019a01e9-8e2e-7e77-9c37-06f7cbd205eb failure=Failure { failure: Some(Failure { message: "Cannot access threading.local.__mro_entries__ from inside a workflow. If this is code from a module not used in a workflow or known to only be used deterministically from a workflow, mark the import as pass through.", source: "", stack_trace: " File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py\", line 418, in activate\n self._run_once(check_conditions=index == 1 or index == 2)\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py\", line 2176, in _run_once\n raise self._current_activation_error\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py\", line 2194, in _run_top_level_workflow_function\n await coro\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py\", line 986, in run_workflow\n result = await self._inbound.execute_workflow(input)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py\", line 2578, in execute_workflow\n return await input.run_fn(*args)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Library/Application Support/JetBrains/PyCharm2025.2/scratches/scratch_20.py\", line 58, in run\n result = await hitl_temporal_agent.run(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_agent.py\", line 348, in run\n return await super().run(\n ^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/abstract.py\", line 206, in run\n async with self.iter(\n\n File \"/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py\", line 210, in __aenter__\n return await anext(self.gen)\n ^^^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_agent.py\", line 727, in iter\n async with super().iter(\n\n File \"/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py\", line 210, in __aenter__\n return await anext(self.gen)\n ^^^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/wrapper.py\", line 194, in iter\n async with self.wrapped.iter(\n\n File \"/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py\", line 210, in __aenter__\n return await anext(self.gen)\n ^^^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/__init__.py\", line 662, in iter\n async with toolset:\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/toolsets/combined.py\", line 52, in __aenter__\n async with self._enter_lock:\n ^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/functools.py\", line 995, in __get__\n val = self.func(instance)\n ^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/toolsets/combined.py\", line 41, in _enter_lock\n return anyio.Lock()\n ^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_synchronization.py\", line 151, in __new__\n return get_async_backend().create_lock(fast_acquire=fast_acquire)\n ^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py\", line 164, in get_async_backend\n module = import_module(f\"anyio._backends._{asynclib_name}\")\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/importlib/__init__.py\", line 90, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"<frozen importlib._bootstrap>\", line 1387, in _gcd_import\n\n File \"<frozen importlib._bootstrap>\", line 1360, in _find_and_load\n\n File \"<frozen importlib._bootstrap>\", line 1310, in _find_and_load_unlocked\n\n File \"<frozen importlib._bootstrap>\", line 488, in _call_with_frames_removed\n\n File \"<frozen importlib._bootstrap>\", line 1387, in _gcd_import\n\n File \"<frozen importlib._bootstrap>\", line 1360, in _find_and_load\n\n File \"<frozen importlib._bootstrap>\", line 1310, in _find_and_load_unlocked\n\n File \"<frozen importlib._bootstrap>\", line 488, in _call_with_frames_removed\n\n File \"<frozen importlib._bootstrap>\", line 1387, in _gcd_import\n\n File \"<frozen importlib._bootstrap>\", line 1360, in _find_and_load\n\n File \"<frozen importlib._bootstrap>\", line 1331, in _find_and_load_unlocked\n\n File \"<frozen importlib._bootstrap>\", line 935, in _load_unlocked\n\n File \"<frozen importlib._bootstrap_external>\", line 995, in exec_module\n\n File \"<frozen importlib._bootstrap>\", line 488, in _call_with_frames_removed\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/__init__.py\", line 3, in <module>\n from ._core._eventloop import current_time as current_time\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py\", line 497, in __call__\n return self.current(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py\", line 248, in _import\n mod = importlib.__import__(name, globals, locals, fromlist, level)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"<frozen importlib._bootstrap>\", line 1470, in __import__\n\n File \"<frozen importlib._bootstrap>\", line 1387, in _gcd_import\n\n File \"<frozen importlib._bootstrap>\", line 1360, in _find_and_load\n\n File \"<frozen importlib._bootstrap>\", line 1331, in _find_and_load_unlocked\n\n File \"<frozen importlib._bootstrap>\", line 935, in _load_unlocked\n\n File \"<frozen importlib._bootstrap_external>\", line 995, in exec_module\n\n File \"<frozen importlib._bootstrap>\", line 488, in _call_with_frames_removed\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py\", line 11, in <module>\n import sniffio\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py\", line 497, in __call__\n return self.current(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py\", line 248, in _import\n mod = importlib.__import__(name, globals, locals, fromlist, level)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"<frozen importlib._bootstrap>\", line 1466, in __import__\n\n File \"<frozen importlib._bootstrap>\", line 1387, in _gcd_import\n\n File \"<frozen importlib._bootstrap>\", line 1360, in _find_and_load\n\n File \"<frozen importlib._bootstrap>\", line 1331, in _find_and_load_unlocked\n\n File \"<frozen importlib._bootstrap>\", line 935, in _load_unlocked\n\n File \"<frozen importlib._bootstrap_external>\", line 995, in exec_module\n\n File \"<frozen importlib._bootstrap>\", line 488, in _call_with_frames_removed\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/sniffio/__init__.py\", line 12, in <module>\n from ._impl import (\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py\", line 497, in __call__\n return self.current(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py\", line 248, in _import\n mod = importlib.__import__(name, globals, locals, fromlist, level)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n File \"<frozen importlib._bootstrap>\", line 1470, in __import__\n\n File \"<frozen importlib._bootstrap>\", line 1387, in _gcd_import\n\n File \"<frozen importlib._bootstrap>\", line 1360, in _find_and_load\n\n File \"<frozen importlib._bootstrap>\", line 1331, in _find_and_load_unlocked\n\n File \"<frozen importlib._bootstrap>\", line 935, in _load_unlocked\n\n File \"<frozen importlib._bootstrap_external>\", line 995, in exec_module\n\n File \"<frozen importlib._bootstrap>\", line 488, in _call_with_frames_removed\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/sniffio/_impl.py\", line 11, in <module>\n class _ThreadLocal(threading.local):\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_restrictions.py\", line 1016, in __getattribute__\n state.assert_child_not_restricted(__name)\n\n File \"/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_restrictions.py\", line 852, in assert_child_not_restricted\n raise RestrictedWorkflowAccessError(\n", encoded_attributes: None, cause: None, failure_info: Some(ApplicationFailureInfo(ApplicationFailureInfo { r#type: "RestrictedWorkflowAccessError", non_retryable: false, details: None, next_retry_delay: None, category: Unspecified })) }), force_cause: Unspecified } from ._impl import ( File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 497, in __call__ return self.current(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 248, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1470, in __import__ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/sniffio/_impl.py", line 11, in <module> class _ThreadLocal(threading.local): File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_restrictions.py", line 1016, in __getattribute__ state.assert_child_not_restricted(__name) File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_restrictions.py", line 852, in assert_child_not_restricted raise RestrictedWorkflowAccessError( temporalio.worker.workflow_sandbox._restrictions.RestrictedWorkflowAccessError: Cannot access threading.local.__mro_entries__ from inside a workflow. If this is code from a module not used in a workflow or known to only be used deterministically from a workflow, mark the import as pass through.
If you add the following change, which makes temporal behave better but not as desired, you still get an error.
diff --git a/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/__init__.py b/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/__init__.py index 9d3aabf4..2aa4630f 100644 --- a/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/__init__.py +++ b/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/__init__.py @@ -67,6 +67,7 @@ class PydanticAIPlugin(ClientPlugin, WorkerPlugin): # Imported inside `logfire._internal.json_schema` when running `logfire.info` inside an activity with attributes to serialize 'numpy', 'pandas', + 'sniffio', ), )
Error with the above:
__call__ on threading.local restricted Failed activation on workflow HitlAgentWorkflow with ID HitlAgentWorkflow and run ID 019a01eb-d842-7069-a23f-d142abce9b79 Traceback (most recent call last): File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 162, in get_async_backend return loaded_backends[asynclib_name] ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ KeyError: 'asyncio' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 418, in activate self._run_once(check_conditions=index == 1 or index == 2) File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2176, in _run_once raise self._current_activation_error File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2194, in _run_top_level_workflow_function await coro File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 986, in run_workflow result = await self._inbound.execute_workflow(input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2578, in execute_workflow return await input.run_fn(*args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Library/Application Support/JetBrains/PyCharm2025.2/scratches/scratch_20.py", line 58, in run result = await hitl_temporal_agent.run( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_agent.py", line 348, in run return await super().run( ^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/abstract.py", line 206, in run async with self.iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_agent.py", line 727, in iter async with super().iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/wrapper.py", line 194, in iter async with self.wrapped.iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/__init__.py", line 662, in iter async with toolset: File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/toolsets/combined.py", line 52, in __aenter__ async with self._enter_lock: ^^^^^^^^^^^^^^^^ File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/functools.py", line 995, in __get__ val = self.func(instance) ^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/toolsets/combined.py", line 41, in _enter_lock return anyio.Lock() ^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_synchronization.py", line 151, in __new__ return get_async_backend().create_lock(fast_acquire=fast_acquire) ^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 164, in get_async_backend module = import_module(f"anyio._backends._{asynclib_name}") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/__init__.py", line 3, in <module> from ._core._eventloop import current_time as current_time File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 497, in __call__ return self.current(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 248, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1470, in __import__ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 27, in <module> threadlocals = threading.local() ^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_restrictions.py", line 1038, in __call__ state.assert_child_not_restricted("__call__") File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_restrictions.py", line 852, in assert_child_not_restricted raise RestrictedWorkflowAccessError( temporalio.worker.workflow_sandbox._restrictions.RestrictedWorkflowAccessError: Cannot access threading.local.__call__ from inside a workflow. If this is code from a module not used in a workflow or known to only be used deterministically from a workflow, mark the import as pass through. __call__ on threading.local restricted Failed activation on workflow HitlAgentWorkflow with ID HitlAgentWorkflow and run ID 019a01eb-d842-7069-a23f-d142abce9b79 Traceback (most recent call last): File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 162, in get_async_backend return loaded_backends[asynclib_name] ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ KeyError: 'asyncio' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 418, in activate self._run_once(check_conditions=index == 1 or index == 2) File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2176, in _run_once raise self._current_activation_error File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2194, in _run_top_level_workflow_function await coro File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 986, in run_workflow result = await self._inbound.execute_workflow(input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/_workflow_instance.py", line 2578, in execute_workflow return await input.run_fn(*args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Library/Application Support/JetBrains/PyCharm2025.2/scratches/scratch_20.py", line 58, in run result = await hitl_temporal_agent.run( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_agent.py", line 348, in run return await super().run( ^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/abstract.py", line 206, in run async with self.iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_agent.py", line 727, in iter async with super().iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/wrapper.py", line 194, in iter async with self.wrapped.iter( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/__init__.py", line 662, in iter async with toolset: File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/toolsets/combined.py", line 52, in __aenter__ async with self._enter_lock: ^^^^^^^^^^^^^^^^ File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/functools.py", line 995, in __get__ val = self.func(instance) ^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/pydantic_ai_slim/pydantic_ai/toolsets/combined.py", line 41, in _enter_lock return anyio.Lock() ^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_synchronization.py", line 151, in __new__ return get_async_backend().create_lock(fast_acquire=fast_acquire) ^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 164, in get_async_backend module = import_module(f"anyio._backends._{asynclib_name}") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/__init__.py", line 3, in <module> from ._core._eventloop import current_time as current_time File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 497, in __call__ return self.current(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 248, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1470, in __import__ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py", line 27, in <module> threadlocals = threading.local() ^^^^^^^^^^^^^^^^^ File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_restrictions.py", line 1038, in __call__ state.assert_child_not_restricted("__call__") File "/Users/davidmontague/Programming/pydantic/pydantic-ai/.venv/lib/python3.12/site-packages/temporalio/worker/workflow_sandbox/_restrictions.py", line 852, in assert_child_not_restricted raise RestrictedWorkflowAccessError( temporalio.worker.workflow_sandbox._restrictions.RestrictedWorkflowAccessError: Cannot access threading.local.__call__ from inside a workflow. If this is code from a module not used in a workflow or known to only be used deterministically from a workflow, mark the import as pass through.
At any rate, the point is that we don't actually use the lock outside of temporal activities when using temporal agents, so using a cached_property just ensures that the creation of the lock doesn't happen until you are in an activity. If we can fix this, great, but if we can't, I don't see a problem with using a cached_property. Is there a downside?
@agronholm
agronholm
Oct 20, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there was a bug, I would like to know what that bug was all about – like what exception was it. And yes, I think this should be an attribute indeed. That's what I really meant.
@agronholm
agronholm
Oct 20, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, so it seems like Temporal patches the stdlib threading module to prevent the use of threadlocals? Using @cached_property seems like an OK workaround then.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, they do lots of crazy stuff to prevent you from doing things that might be "non-deterministic" in their sandbox. (The point is that they need to be able to re-run workflow code from the start where they have cached outputs of any calls to activities, and have the program end in the exact same state it started in, as the way they handle retries, app restarts, etc.).
I don't know much about specifically what temporal thinks about thread locals or why they would be problematic, but yes what you are describing sounds very plausible to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we return a tuple?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed this is a good idea if possible for reasons of variance
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You intend to vendor that function right? Because I don't think we should wait until that's released and then bump the requirement all the way to the latest -- that'd be annoying for users
@agronholm
agronholm
Oct 17, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This catches and ignores CancelledError which is a no-no even on plain asyncio. So I'm not critizing this change, but rather question why it did this in the first place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To not have a noise output on CTRL + C on the terminal.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My suspicion was that under some circumstances a keyboard interrupt was getting converted into a CancelledError and this was meant to handle that. Investigating the git blame has (implicitly) confirmed this — it was introduced here https://github.com/pydantic/pydantic-ai/pull/1317/files#diff-6d44ee5732e793156e7e5f42cdda7759921b19cf80c72ee86803f28989c47646R194 and was converting a KeyboardInterrupt handler into a CancelledError handler.
If you run the cli clai and press ctrl+c while generating a response you can hit this branch. Here's the traceback I get if I add console.print(traceback.format_exc()) in this exception handler on main:
File "pydantic-ai/pydantic_ai_slim/pydantic_ai/_cli.py", line 260, in run_chat messages = await ask_agent(agent, text, stream, console, code_theme, deps, messages) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/pydantic_ai_slim/pydantic_ai/_cli.py", line 295, in ask_agent async with node.stream(agent_run.ctx) as handle_stream: File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/pydantic_ai_slim/pydantic_ai/_agent_graph.py", line 407, in stream async with ctx.deps.model.request_stream( File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/pydantic_ai_slim/pydantic_ai/models/openai.py", line 424, in request_stream response = await self._completions_create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/pydantic_ai_slim/pydantic_ai/models/openai.py", line 487, in _completions_create return await self.client.chat.completions.create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/openai/resources/chat/completions/completions.py", line 2583, in create return await self._post( ^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1794, in post return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1529, in request response = await self._client.send( ^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1629, in send response = await self._send_handling_auth( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1657, in _send_handling_auth response = await self._send_handling_redirects( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects response = await self._send_single_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1730, in _send_single_request response = await transport.handle_async_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 394, in handle_async_request resp = await self._pool.handle_async_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request raise exc from None File "pydantic-ai/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request response = await connection.handle_async_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request raise exc File "pydantic-ai/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 78, in handle_async_request stream = await self._connect(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 156, in _connect stream = await stream.start_tls(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 70, in start_tls ssl_stream = await anyio.streams.tls.TLSStream.wrap( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/anyio/streams/tls.py", line 132, in wrap await wrapper._call_sslobject_method(ssl_object.do_handshake) File "pydantic-ai/.venv/lib/python3.12/site-packages/anyio/streams/tls.py", line 147, in _call_sslobject_method data = await self.transport_stream.receive() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic-ai/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 1246, in receive await self._protocol.read_event.wait() File "/Users/davidmontague/.asdf/installs/python/3.12.3/lib/python3.12/asyncio/locks.py", line 212, in wait await fut asyncio.exceptions.CancelledError
I guess to handle this better we might need to be passing a KeyboardInterrupt up somewhere, but I'm not sure if it's deep in the internals of some other library or something we can control ourselves. But anyway, this is why it suppresses the CancelledError today.
@agronholm
agronholm
Oct 20, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Usually the try...except should be around asyncio.run(), as swallowing CancelledError is bad practice.
@agronholm
agronholm
Oct 20, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not to mention the fact that it doesn't even exit the loop when swallowing the exception.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The point is that we don't want it to exit the loop — we want it to just say "Interrupted" and go back to the prompt where you can type stuff in. If a request isn't running, then we exit when you press ctrl+c. The intent is that if you ask it a question you can press ctrl+c to stop from waiting for the AI to respond and give it a new prompt rather than just exiting the program.
(I'm not saying there isn't a better way to do this, just the current behavior is as desired.)
@agronholm don't worry about removing the use of asyncio.wait etc. related to the agent graph, I've done a massive refactor of that code in #2982 and it now uses anyio rather than asyncio. I suspect attempts to refactor the agent graph code will result in major merge conflicts with that PR. I'm hoping to merge it soon...
@agronholm I noticed the code in #2982 was having issues with that same example temporal script above, but I discovered that adding the following at the top of pydantic_ai_slim/pydantic_ai/durable_exec/temporal/__init__.py makes the error go away:
# We need eagerly import the anyio backends or it will happens inside workflow code and temporal has issues import anyio._backends._asyncio try: import anyio._backends._trio # noqa F401 except ImportError: pass
I believe this is because it results in the relevant subclassing etc. all happening before workflow code gets executed (otherwise the imports happen dynamically and inside workflow code, which triggers the temporal sandbox issues).
Is this likely to be problematic? E.g., are there any side effects to these imports and/or significant performance consequences (e.g. if they are known to be slow to import). Note that this is only triggered by importing temporal, and I think if you are using temporal you are probably willing to pay a little extra initialization performance penalty for this. But I want to make sure this isn't known to cause issues. (I know it's importing private modules but my guess is this not likely to break any time soon..?)
@agronholm I noticed the code in #2982 was having issues with that same example temporal script above, but I discovered that adding the following at the top of
pydantic_ai_slim/pydantic_ai/durable_exec/temporal/__init__.pymakes the error go away:# We need eagerly import the anyio backends or it will happens inside workflow code and temporal has issues import anyio._backends._asyncio try: import anyio._backends._trio # noqa F401 except ImportError: passI believe this is because it results in the relevant subclassing etc. all happening before workflow code gets executed (otherwise the imports happen dynamically and inside workflow code, which triggers the temporal sandbox issues).
Is this likely to be problematic? E.g., are there any side effects to these imports and/or significant performance consequences (e.g. if they are known to be slow to import). Note that this is only triggered by importing temporal, and I think if you are using temporal you are probably willing to pay a little extra initialization performance penalty for this. But I want to make sure this isn't known to cause issues. (I know it's importing private modules but my guess is this not likely to break any time soon..?)
Unfortunate as this is, I don't see it breaking anything in the foreseeable future.
No description provided.