Calling `c.events()` seems to break my python script sometimes

[ShawkMusic]

When I run a python script that uses caldav (more details below), the script freezes for like a minute or two and then I get this error:

Traceback (most recent call last):
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 668, in _error_catcher
    yield
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 771, in _raw_read
    data = self._fp_read(amt) if not fp_closed else b""
           ~~~~~~~~~~~~~^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 756, in _fp_read
    return self._fp.read(amt) if amt is not None else self._fp.read()
                                                      ~~~~~~~~~~~~~^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/backend/_base.py", line 342, in read
    data, self._eot, self.trailers = self.__internal_read_st(
                                     ~~~~~~~~~~~~~~~~~~~~~~~^
        __size, self._stream_id
        ^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/backend/hface.py", line 1544, in __read_st
    events: list[DataReceived | HeadersReceived] = self.__exchange_until(  # type: ignore[assignment]
                                                   ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        DataReceived,
        ^^^^^^^^^^^^^
    ...<5 lines>...
        respect_end_stream_signal=__respect_end_signal,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/backend/hface.py", line 945, in __exchange_until
    data_in = sync_recv_gro(self.sock, self.blocksize)
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/contrib/ssa/_gro.py", line 88, in sync_recv_gro
    data, ancdata, _flags, addr = sock.recvmsg(bufsize, ancbufsize)
                                  ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/models.py", line 1436, in content
    self._content = self.raw.read(decode_content=True)  # type: ignore[arg-type]
                    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 884, in read
    data = self._raw_read(amt)
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 770, in _raw_read
    with self._error_catcher():
         ~~~~~~~~~~~~~~~~~~~^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 673, in _error_catcher
    raise ReadTimeoutError(self._pool, None, "Read timed out.") from e  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3_future.exceptions.ReadTimeoutError: None: Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File [REDACTED (path to python script running the code)], line 296, in <module>
    for event in c.events():
                 ~~~~~~~~^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/collection.py", line 1116, in events
    return self.search(comp_class=Event)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/collection.py", line 965, in search
    return my_searcher.search(
           ~~~~~~~~~~~~~~~~~~^
        self, server_expand, split_expanded, props, xml, post_filter, _hacks
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/search.py", line 480, in search
    (response, objects) = calendar._request_report_build_resultlist(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        xml, self.comp_class, props=props
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/collection.py", line 784, in _request_report_build_resultlist
    response = self._query(xml, 1, "report")
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/davobject.py", line 219, in _query
    ret = getattr(self.client, query_method)(url, body, depth)
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/davclient.py", line 873, in report
    return self.request(
           ~~~~~~~~~~~~^
        url,
        ^^^^
    ...<2 lines>...
        {"Depth": str(depth), "Content-Type": 'application/xml; charset="utf-8"'},
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/davclient.py", line 1034, in request
    log.debug("server responded with %i %s" % (r.status_code, r.reason))
                                               ^^^^^^^^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/models.py", line 1049, in __getattribute__
    super().__getattribute__("_gather")()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/models.py", line 1035, in _gather
    super().__getattribute__("connection").gather(self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/adapters.py", line 1315, in gather
    next_resp = self._future_handler(response, low_resp)
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/adapters.py", line 1148, in _future_handler
    response.content
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/models.py", line 1053, in __getattribute__
    return super().__getattribute__(item)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/models.py", line 1444, in content
    raise ConnectionError(e)
niquests.exceptions.ConnectionError: None: Read timed out.

Trying to run a minimal reproduction on the python shell gives me this:

>>> with caldav.DAVClient(
...         url="[REDACTED]",
...         username="[REDACTED]",
...         password="[REDACTED]",
...     ) as client:
...         my_principal = client.principal()
...         calendars = my_principal.calendars()
...         
...         for c in calendars:
...             if c.url == "[REDACTED]":
...                 for event in c.events():
...                     print("bruh")
...                     

Traceback (most recent call last):
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 668, in _error_catcher
    yield
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 771, in _raw_read
    data = self._fp_read(amt) if not fp_closed else b""
           ~~~~~~~~~~~~~^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 756, in _fp_read
    return self._fp.read(amt) if amt is not None else self._fp.read()
                                                      ~~~~~~~~~~~~~^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/backend/_base.py", line 342, in read
    data, self._eot, self.trailers = self.__internal_read_st(
                                     ~~~~~~~~~~~~~~~~~~~~~~~^
        __size, self._stream_id
        ^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/backend/hface.py", line 1544, in __read_st
    events: list[DataReceived | HeadersReceived] = self.__exchange_until(  # type: ignore[assignment]
                                                   ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        DataReceived,
        ^^^^^^^^^^^^^
    ...<5 lines>...
        respect_end_stream_signal=__respect_end_signal,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/backend/hface.py", line 945, in __exchange_until
    data_in = sync_recv_gro(self.sock, self.blocksize)
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/contrib/ssa/_gro.py", line 88, in sync_recv_gro
    data, ancdata, _flags, addr = sock.recvmsg(bufsize, ancbufsize)
                                  ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/models.py", line 1436, in content
    self._content = self.raw.read(decode_content=True)  # type: ignore[arg-type]
                    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 884, in read
    data = self._raw_read(amt)
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 770, in _raw_read
    with self._error_catcher():
         ~~~~~~~~~~~~~~~~~~~^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/urllib3_future/response.py", line 673, in _error_catcher
    raise ReadTimeoutError(self._pool, None, "Read timed out.") from e  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3_future.exceptions.ReadTimeoutError: None: Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<python-input-11>", line 14, in <module>
    for event in c.events():
                 ~~~~~~~~^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/collection.py", line 1116, in events
    return self.search(comp_class=Event)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/collection.py", line 965, in search
    return my_searcher.search(
           ~~~~~~~~~~~~~~~~~~^
        self, server_expand, split_expanded, props, xml, post_filter, _hacks
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/search.py", line 480, in search
    (response, objects) = calendar._request_report_build_resultlist(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        xml, self.comp_class, props=props
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/collection.py", line 784, in _request_report_build_resultlist
    response = self._query(xml, 1, "report")
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/davobject.py", line 219, in _query
    ret = getattr(self.client, query_method)(url, body, depth)
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/davclient.py", line 873, in report
    return self.request(
           ~~~~~~~~~~~~^
        url,
        ^^^^
    ...<2 lines>...
        {"Depth": str(depth), "Content-Type": 'application/xml; charset="utf-8"'},
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/caldav/davclient.py", line 1034, in request
    log.debug("server responded with %i %s" % (r.status_code, r.reason))
                                               ^^^^^^^^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/models.py", line 1049, in __getattribute__
    super().__getattribute__("_gather")()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/models.py", line 1035, in _gather
    super().__getattribute__("connection").gather(self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/adapters.py", line 1315, in gather
    next_resp = self._future_handler(response, low_resp)
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/adapters.py", line 1148, in _future_handler
    response.content
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/models.py", line 1053, in __getattribute__
    return super().__getattribute__(item)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/nix/store/n3a01p8l886s3v5n4x7frg52y3hppnfs-python3-3.13.12-env/lib/python3.13/site-packages/niquests/models.py", line 1444, in content
    raise ConnectionError(e)

If I remove the line for event in c.events():, it works just fine

>>> with caldav.DAVClient(
...         url="[REDACTED]",
...         username="[REDACTED]",
...         password="[REDACTED]",
...     ) as client:
...         my_principal = client.principal()
...         calendars = my_principal.calendars()
...         
...         for c in calendars:
...             if c.url == "[REDACTED]":
...                 print("bruh")
...                 
bruh

There doesn't seem to be a difference if I link to different calendars either.
Puzzlingly, sometimes it just decides it wants to work once in a while even with the for event in c.events(): line, so I have no idea what's really going on, but including the for event in c.events(): seems to be the thing that most consistently makes it break, as far as I can tell.

I'm using NixOS 26.05

[tobixen]

I have on my list to try replacing Archlinux with NixOS on some of my systems :-)

What I notice from the error logs is that it fails on timeout. Does it take a long time to execute?

One thing you may try is to replace the niquests library with the requests library. This involves removing the niquests library and installing the requests library. If you're running in a virtual environment, sed -i -e 's/niquests/requests/' pyproject.toml should do the trick.

I should try to look into how to deal with the gross amount of During handling of the above exception, another exception occurred, that's not nice.

[tobixen]

I came to think, the default timeouts for niquests is relatively low compared to "indefinite" in requests. So if your server is just slow, things may be tweaked by adjusting the timeout parameter when constructing davclient.

PS: https://caldav.readthedocs.io/latest/v3-migration.html

[tobixen]

⚠️ This comment is AI-generated (Claude Sonnet 4.6 via Claude Code) on behalf of tobixen

I looked into the root cause of this.

Why it's intermittent

niquests uses HTTP/2 by default and reuses persistent connections. When a connection is stale (the server closed it, but the OS still considers it alive), the next request over that connection will fail with a timeout. Sometimes a fresh connection is established (works fine), sometimes a stale one is reused (fails). This explains the "sometimes works" behavior.

Why the error appears at r.status_code access, not at the request itself

niquests uses a lazy/deferred response-gathering model. session.request() sends the request but may not fully receive the response. The response is only gathered ("materialized") when you first access an attribute like r.status_code or r.headers. This is what triggers the _gather() call visible in the traceback. So the ConnectionError is raised inside a log.debug(...) call or similar — hence the confusing "During handling of the above exception, another exception occurred" chain.

The workaround (as suggested above)

Replacing niquests with requests avoids the issue because requests uses HTTP/1.1 keep-alive rather than HTTP/2 multiplexing, and raises exceptions eagerly at request time rather than lazily on attribute access.

What should be fixed in the library

1. _sync_request should catch requests.exceptions.ConnectionError and requests.exceptions.Timeout and re-raise them with a clearer message (avoiding the nested exception noise).
2. For idempotent methods (REPORT, PROPFIND, GET — used by c.events()), it would make sense to retry once on ConnectionError, which would transparently recover from stale connection reuse.

I'll look into implementing this.

⚠️ This comment is AI-generated (Claude Sonnet 4.6 via Claude Code) on behalf of tobixen

[ShawkMusic]

I have on my list to try replacing Archlinux with NixOS on some of my systems :-)

I just moved from my Arch system a few months ago, actually! It feels much cleaner to have a consistent config file that will always rebuild the exact same system instead of a million random commands I typed years ago that I completely forgot about.

What I notice from the error logs is that it fails on timeout. Does it take a long time to execute?

Yeah, it takes like a few minutes untill it gives the error message.

I should note that this started happening like today or yesterday, and it was working just fine before that. I've been running pretty much the same script for like a year now (little changes here and there, but I don't think anything that would cause this to happen) and haven't had this specific issue before, so I'm pretty sure this is probably because of a bug from something that was changed recently, either in python-caldav or that niquests library you mentioned. I could actually rollback my NixOS system to check and make sure if you want.

One thing you may try is to replace the niquests library with the requests library. This involves removing the niquests library and installing the requests library. If you're running in a virtual environment, sed -i -e 's/niquests/requests/' pyproject.toml should do the trick.

I generally don't like to tamper with things in my system that I don't fully understand unless I really need to, and this seems like a bug that needs to be fixed in the code anyway for the reasons I listed above. But I'll keep it in mind.

[tobixen]

I should note that this started happening like today or yesterday, and it was working just fine before that.

I've been releasing 3.0, 3.0.1, 3.0.2 and 3.1 rapidly over the last week or so - but the niquest library was there in 2.2.6 as well.

I've noticed some few releases of niquests over the last weeks.

I'll investigate a bit more. Claude claims that the problem may be because the server closed the TCP-connection - in that case I think perhaps a TimeoutError on the client side is inappropriate.

[tobixen]

I'd be happy if you could test the branch issue647.

[tobixen]

If the hypothesis by Claude is correct, the annoying waiting times will disappear if setting timeout to a very low value, i.e. 5 or 10 (depedent on what actually works out in your setup). Could you try that, @ShawkMusic ?

[tobixen]

It feels much cleaner to have a consistent config file that will always rebuild the exact same system instead of a million random commands I typed years ago that I completely forgot about.

Tell me about it ... my day-job is a sysadmin. "CLI-ops" or "click-ops" where someone have been rolling out servers, databases and configuration directly from the command line or from some web-UI is a nightmare when things should be migrated or upgraded (and still it's an upgrade from back in the days where each database or application server needed a dedicated physical server). We're typically managing VMs by Puppet, and nowadays we've become pretty good at it, but I do remember we had major issues upgrading some "unicorn"-servers some ten-fifteen years ago. The nix config files seems to make Puppet completely redundant. But I digress ... again, sorry :-)

[tobixen]

Ref jawah/niquests#352 the niquest/urllib3-developer says that Claude is hallucinating when it found the "root cause" for this issue. He's requesting more logs - see jawah/niquests#352 (comment)

I'd be happy if you could test out the caldav branch issue647 - which Claude claims is a workaround for the issue it found.

I'd also be happy if you could play a bit with the timeout settings.