Skip to content

Handle the loop closing between ThreadsafeProxy's is_closed() check and dispatch - #741

Open
zigpy-review-bot wants to merge 3 commits into
devfrom
zigpy-bot/threadsafe-proxy-closed-loop-race
Open

Handle the loop closing between ThreadsafeProxy's is_closed() check and dispatch#741
zigpy-review-bot wants to merge 3 commits into
devfrom
zigpy-bot/threadsafe-proxy-closed-loop-race

Conversation

@zigpy-review-bot

@zigpy-review-bot zigpy-review-bot commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Closes #740. Follow-up to #727 and #739 — this closes out the last members of the RuntimeError: Event loop is closed teardown-race family.

ThreadsafeProxy.func_wrapper snapshots the loop object at construction, so #739's ordering fix (publish self.loop = None before closing) can't reach it: the worker thread can close the loop between the proxy's loop.is_closed() check and the dispatch via asyncio.run_coroutine_threadsafe() / loop.call_soon_threadsafe(), and both raise RuntimeError: Event loop is closed at the caller. This wraps both dispatches in try/except RuntimeError and treats close-during-dispatch exactly like already-closed: log the existing "Attempted to use a closed event loop" warning and drop the call. The except is narrow — both try blocks wrap only the scheduling call (for the async path, call() merely constructs the coroutine; its body never runs there), so no RuntimeError from user code can be swallowed.

Two findings from pre-open review (details below) are folded in:

  • Disconnected async proxy calls are now awaitable. The pre-existing is_closed() branch returned bare None for coroutine methods too, so real callers that await through a proxy (await self._gw.reset(), await self._gw.disconnect(), await self._gw.send_data(...)) got TypeError: object NoneType can't be used in 'await' expression when disconnected — trading one teardown exception for another. Both disconnected paths (already-closed and closed-mid-dispatch) now return an already-completed future for coroutine methods, so await proxy.method() resolves cleanly to None; sync calls still return None as before. On the race path the never-to-be-scheduled coroutine is explicitly closed, so it doesn't warn as never-awaited.
  • EventLoopThread.run_coroutine_threadsafe() raises legibly when the loop is gone. After Clear EventLoopThread.loop before closing it #739 the shutdown-window failure there was AttributeError: 'NoneType' object has no attribute 'call_soon_threadsafe'; it now snapshots self.loop and raises RuntimeError("Event loop is not running"). The sole caller (uart.connect) already handles this via except Exception.

Two follow-up commits, from post-open review of the same family:

  • The passed coroutine is closed on both of run_coroutine_threadsafe()'s failure paths. The None guard closes it before raising, and when the worker closes the loop after the snapshot (so asyncio.run_coroutine_threadsafe() itself raises RuntimeError), it is now closed before re-raising too — mirroring the proxy's async dispatch path — so neither failure mode leaks a "coroutine ... was never awaited" RuntimeWarning at teardown.
  • inspect.iscoroutinefunction() replaces the deprecated asyncio.iscoroutinefunction() (deprecated since Python 3.14, removal slated for 3.16; these were bellows' only two call sites). Drop-in for every case here: real coroutine functions, plain callables, functools.partial, AsyncMock.

Tests

Five new tests. test_proxy_loop_closed_during_async_dispatch and test_proxy_loop_closed_during_sync_dispatch pin the race window by making loop.call_soon_threadsafe raise RuntimeError while is_closed() still returns False — a faithful stand-in, since a genuinely closed loop would trip the earlier guard. test_proxy_loop_closed_async pins the awaitable disconnected result on the already-closed branch; test_thread_run_coroutine_threadsafe_loop_not_running pins the RuntimeError on the None path; test_thread_run_coroutine_threadsafe_loop_closed_mid_dispatch pins the coroutine being closed (CORO_CLOSED) on the post-snapshot race path. All five fail on dev and pass with the change.

Verification
  • pytest tests/446 passed (Python 3.14.5), clean under -W error::RuntimeWarning -W error::DeprecationWarning (pins both the no-leaked-coroutine behavior and the deprecation removal)
  • With bellows/thread.py reverted to dev: the five new tests fail (leaked RuntimeError ×2, TypeError on await None, AttributeError, leaked never-awaited coroutine), rest pass
  • Pre-open review: an Opus subagent review (confirmed the except RuntimeError breadth is safe and flagged the coroutine leak on the raise path) and a Copilot CLI second opinion (GPT-5.6 Sol; flagged the await None TypeError) — both findings addressed as described
  • pre-commit: autoflake, black, flake8, isort, codespell, ruff pass; pyupgrade/mypy fail on untouched files in the local hook env only (same Python 3.14 breakage as noted on Clear EventLoopThread.loop before closing it #739) — CI is the real check

… and dispatch

The worker thread can close the loop after `func_wrapper`'s `is_closed()`
check but before `run_coroutine_threadsafe()` / `call_soon_threadsafe()`,
raising `RuntimeError: Event loop is closed` at the caller -- the same
race #727 suppressed in `force_stop()`. Treat close-during-dispatch like
already-closed: log the existing warning and drop the call.

Disconnected async proxy calls (both already-closed and closed-mid-
dispatch) now return an already-completed future resolving to None
instead of bare None, so `await proxy.method()` no longer raises
`TypeError: object NoneType can't be used in 'await' expression`.

Also raise a legible `RuntimeError` from
`EventLoopThread.run_coroutine_threadsafe()` when the loop is already
gone, instead of `AttributeError: 'NoneType' object has no attribute
'call_soon_threadsafe'`, closing the passed coroutine so it doesn't
warn as never-awaited.

Closes #740
@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.55%. Comparing base (553ec35) to head (6905629).

Additional details and impacted files
@@           Coverage Diff           @@
##              dev     #741   +/-   ##
=======================================
  Coverage   99.55%   99.55%           
=======================================
  Files          64       64           
  Lines        4263     4286   +23     
=======================================
+ Hits         4244     4267   +23     
  Misses         19       19           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

`asyncio.iscoroutinefunction()` is deprecated since Python 3.14 and
slated for removal in 3.16; `inspect.iscoroutinefunction()` is a
drop-in replacement for every case here (coroutine functions, plain
callables, `functools.partial`, `AsyncMock`).
`EventLoopThread.run_coroutine_threadsafe`'s `None` guard covers the
already-exited worker, but when the worker closes the loop after the
snapshot, `asyncio.run_coroutine_threadsafe()` raises `RuntimeError`
correctly while the passed coroutine leaks as never-awaited. Close it
before re-raising, mirroring the proxy's async dispatch path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ThreadsafeProxy can still raise RuntimeError: Event loop is closed — remaining check-then-use race after #727/#739

1 participant