Await connection callback instead of creating a detached task#61
Open
lnagel wants to merge 1 commit intoicgood:mainfrom
Open
Await connection callback instead of creating a detached task#61lnagel wants to merge 1 commit intoicgood:mainfrom
lnagel wants to merge 1 commit intoicgood:mainfrom
Conversation
_read_then_call used asyncio.create_task() to fire off the connection callback without keeping a reference to the resulting task. Since the event loop only holds weak references to tasks, the GC can collect them at any time, producing "Task was destroyed but it is pending!" errors under load. Because asyncio.start_server already runs _read_then_call in a strongly-referenced task, the callback can simply be awaited directly. This keeps the connection handler alive for the full connection lifetime without needing a separate task. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ProxyProtocolReader._read_then_callfires off the user's connection callback viaasyncio.create_task()without saving a reference to the returned task:Python's event loop tracks tasks in a
WeakSet, meaning tasks with no strong reference elsewhere are eligible for garbage collection at any time — even mid-execution. Theasyncio.create_task()docs warn about this explicitly:This is CPython issue #91887, which remains unresolved as of Python 3.14. The Ruff linter includes rule RUF006 (
asyncio-dangling-task) specifically to catch this pattern.Under load (thousands of concurrent connections), the GC will collect these unreferenced tasks, producing:
Why
asyncio.start_serverconnections are not affectedasyncio.start_serveralready creates a task per connection, and since CPython gh-90467 (backported to 3.10/3.11),StreamReaderProtocolkeeps a strong reference to the handler task. The reference chain is: Server → Transport → StreamReaderProtocol → Task → coroutine.The nested task spawned inside
_read_then_calllacks this protection. Once_read_then_callreturns (immediately aftercreate_task), the inner task has no strong reference and is vulnerable to GC.Fix
Replace
asyncio.create_task(callback(...))withawait callback(...).This is the right fix (as opposed to saving task references to a set) because:
asyncio.start_serveralready runs each connection in its own task. Connections are concurrent with each other regardless. The nestedcreate_task()only added unnecessary concurrency within a single connection's lifecycle.await, exceptions from the callback propagate naturally. With a detached task, exceptions are silently swallowed — a second compounding bug.StreamReaderProtocolwould see the handler return immediately after header parsing, even though the real work was still running in an untracked task.asyncio.start_serverdocs, whereclient_connected_cbdoes its work directly.Test plan