Connection pool has no acquire timeout, liveness check, idle eviction or max lifetime

[Yaraslaut]

DataMapper/Pool.hpp has a more sophisticated waiter model than any C++ peer — a single FIFO shared between sync and async waiters, three growth strategies selected at compile time — but none of the operational controls that keep a pool healthy in production.

Missing

Control	Status	Evidence
Acquire timeout	absent	Pool.hpp:269 — node->cv.wait(lock, pred), no wait_for. SqlBackup/ConnectionPool.cpp:46-53 — same. There is no wait_for/wait_until anywhere in src/Lightweight.
Validation on borrow	absent	SqlConnection::IsAlive() exists (SqlConnection.cpp:471) but no pool calls it
Idle eviction	absent	idle connections are held indefinitely
Max lifetime	absent	a connection is never proactively recycled
Retry on broken connection	absent	a dead connection is handed to the caller as-is

Consequences

• A connection killed by a firewall idle timeout, a failover, or a server restart is handed to the next caller, which fails with a driver error the pool could have prevented.
• An exhausted BoundedWait pool blocks the calling thread forever with no diagnostic.

Prior art

ormpp's much simpler pool does ping() on checkout and recycles at 8 hours (ormpp/connection_pool.hpp); HikariCP-class pools do all five.

Suggested scope

1. Acquire(std::chrono::milliseconds timeout) returning std::expected<PooledDataMapper, PoolError> — pairs naturally with Add deadline propagation across queries, transactions, and retries #553 (deadline propagation).
2. Optional validation-on-borrow using the existing IsAlive(), off by default for latency-sensitive callers.
3. maxIdleTime / maxLifetime in PoolConfig.

Items 1 and 3 are PoolConfig additions, which is an NTTP, so they are source-compatible for existing users of the default config.

[Yaraslaut]

Follow-up: OS-level connection limits and unbounded concurrency

Discussion while implementing this (#575) surfaced a related gap worth recording separately, since it is out of scope here but arguably more dangerous than any of the five controls listed above.

The pool's only hard concurrency cap is BoundedWait

_checkedOut — the counter enforcing maxSize — is maintained solely for GrowthStrategy::BoundedWait. For BoundedOverflow and UnboundedGrow, Acquire() creates a fresh DataMapper whenever no usable idle one exists, with no ceiling:

auto entry = TakeIdleLocked(retired);
if (!entry.mapper)
    return PooledDataMapper(*this, MakeEntry());   // no capacity check

For BoundedOverflow — the default DataMapperPool — maxSize bounds only the idle set, not live connections. So N concurrent callers open N connections. Under a burst, load is converted into socket count until something outside the process breaks.

What actually breaks, and in what order

The ceilings are all external, so the first symptom is a driver error rather than back-pressure:

1. Database max_connections — usually the tightest. PostgreSQL defaults to 100, some reserved for superusers.
2. File descriptors per process — RLIMIT_NOFILE, commonly 1024 inside a container. Note that on macOS kern.maxfilesperproc caps a process independently of ulimit -n, which can report a much larger number and mislead.
3. Ephemeral ports — a TCP connection is identified by its 4-tuple, and against a single database endpoint only the source port varies. That caps concurrent connections to the size of the ephemeral range (~28k on Linux, ~16k on macOS) regardless of how many fds are available.
4. TIME_WAIT — a closed connection holds its source port for 2×MSL (~60s). This converts the port count into a rate limit of roughly 270–470 new connections/second sustained, past which connects fail with "address already in use".

Point 4 is the one that interacts with the maxIdleTime / maxLifetime bounds added in #575: retirement deliberately recreates connections. At sane values it is negligible (16 connections on a 30-minute lifetime is ~0.01/s, five orders of magnitude below the limit), but a pathologically small idle bound on a BoundedOverflow pool turns a pooled workload back into connect-per-query and can approach it.

Related fix already in #575

BoundedWait used to increment _checkedOut before constructing the DataMapper:

++_checkedOut;
return PooledDataMapper(*this, std::make_unique<DataMapper>());  // throws on EMFILE

Under fd or port exhaustion the constructor throws and the slot was consumed permanently, so a pool that hit an OS limit shrank on every failure until it deadlocked with zero connections and every caller blocked forever. #575 claims the slot only after the connection stands up, so the pool now recovers once the pressure clears. This failure mode is only reachable under exactly the exhaustion described above.

Proposed follow-ups

1. Bound concurrency for the non-waiting strategies, or at least emit a SqlLogger warning when a BoundedOverflow pool's live count runs far past maxSize. Today the first sign of trouble is EMFILE or max_connections, not back-pressure.
2. Reconsider the default strategy. BoundedOverflow is reasonable for CLI tools and batch jobs and wrong for servers, where it trades a bounded wait for unbounded sockets. Changing the default is behaviour-affecting and needs its own discussion.
3. Document server sizing guidance — choose BoundedWait; size maxSize from (max_connections − reserved) / instances and keep it near cores × 2 rather than large; use Acquire(timeout) and shed the request on timeout; count timeouts as the saturation signal; raise LimitNOFILE as headroom rather than as the mechanism. This part is small enough to fold into feat(Pool): acquire timeout, borrow validation and connection age bounds #575.

Items 1 and 2 change behaviour for existing users and probably deserve their own issue; opening this comment here first to keep the context together.