Skip to content

Await connection callback instead of creating a detached task#61

Open
lnagel wants to merge 1 commit intoicgood:mainfrom
namespace-ee:fix/await-callback-instead-of-detached-task
Open

Await connection callback instead of creating a detached task#61
lnagel wants to merge 1 commit intoicgood:mainfrom
namespace-ee:fix/await-callback-instead-of-detached-task

Conversation

@lnagel
Copy link

@lnagel lnagel commented Feb 27, 2026

Problem

ProxyProtocolReader._read_then_call fires off the user's connection callback via asyncio.create_task() without saving a reference to the returned task:

asyncio.create_task(callback(reader, writer, sock_info))

Python's event loop tracks tasks in a WeakSet, meaning tasks with no strong reference elsewhere are eligible for garbage collection at any time — even mid-execution. The asyncio.create_task() docs warn about this explicitly:

Important: Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn't referenced elsewhere may get garbage collected at any time, even before it's done.

This is CPython issue #91887, which remains unresolved as of Python 3.14. The Ruff linter includes rule RUF006 (asyncio-dangling-task) specifically to catch this pattern.

Under load (thousands of concurrent connections), the GC will collect these unreferenced tasks, producing:

Task was destroyed but it is pending!

Why asyncio.start_server connections are not affected

asyncio.start_server already creates a task per connection, and since CPython gh-90467 (backported to 3.10/3.11), StreamReaderProtocol keeps a strong reference to the handler task. The reference chain is: Server → Transport → StreamReaderProtocol → Task → coroutine.

The nested task spawned inside _read_then_call lacks this protection. Once _read_then_call returns (immediately after create_task), the inner task has no strong reference and is vulnerable to GC.

Fix

Replace asyncio.create_task(callback(...)) with await callback(...).

This is the right fix (as opposed to saving task references to a set) because:

  1. No concurrency is lost. asyncio.start_server already runs each connection in its own task. Connections are concurrent with each other regardless. The nested create_task() only added unnecessary concurrency within a single connection's lifecycle.
  2. Proper exception propagation. With await, exceptions from the callback propagate naturally. With a detached task, exceptions are silently swallowed — a second compounding bug.
  3. Clean connection lifecycle. The handler coroutine now runs for the full lifetime of the connection. Previously, the StreamReaderProtocol would see the handler return immediately after header parsing, even though the real work was still running in an untracked task.
  4. Idiomatic. This matches the pattern shown in the asyncio.start_server docs, where client_connected_cb does its work directly.

Test plan

  • Existing test suite passes
  • 100% code coverage maintained

_read_then_call used asyncio.create_task() to fire off the connection
callback without keeping a reference to the resulting task. Since the
event loop only holds weak references to tasks, the GC can collect
them at any time, producing "Task was destroyed but it is pending!"
errors under load.

Because asyncio.start_server already runs _read_then_call in a
strongly-referenced task, the callback can simply be awaited directly.
This keeps the connection handler alive for the full connection
lifetime without needing a separate task.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@lnagel lnagel marked this pull request as ready for review February 27, 2026 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant