Skip to content

Handle asynchronous cancellations for HTTPConnection instances #785

Closed
@TheTechromancer

Description

@TheTechromancer

The gist of the race condition is that when a web request is cancelled within a specific time window, it will degrade the state of the connection pool, leaving it unable to issue further requests. It also seems to create a deadlock.

It can be reproduced with the following code:

import trio
import httpcore


TIMEOUT = {"read": 5, "write": 5, "connect": 5, "pool": 5}


async def send_request(client, seen_response):
    response = await client.request("GET", "http://127.0.0.1:8000", extensions={"timeout": TIMEOUT})

    # Print the first response that we see and set the "seen_response" flag.
    if not seen_response.is_set():
        print(response)
        seen_response.set()


async def main():
    async with httpcore.AsyncConnectionPool() as client:
        while 1:
            async with trio.open_nursery() as nursery:
                # This event is used to signal the first response we recieve.
                seen_response = trio.Event()

                # Kick off one hundred HTTP requests.
                for idx in range(100):
                    nursery.start_soon(send_request, client, seen_response)
                
                # As soon as we see a response we can cancel out of the nursery,
                # rather than allowing all tasks to complete.
                await seen_response.wait()
                nursery.cancel_scope.cancel()

trio.run(main)

This should produce a httpcore.PoolTimeout error on the second or third iteration, after which the loop enters into a deadlock.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions