connected: add incremental connectivity check

[spkrka]

This series adds an incremental connectivity check to
check_connected(), gated behind transfer.connectivityCheck=incremental
(so no expected changes unless you opt-in to it).

It relates to the RFC I sent out earlier:

[RFC] check_connected: toward incoming-proportional cost
https://lore.kernel.org/git/CAL71e4Nf=-zCrfN7ghEVGq11irajJhtdxYZgKe0Ycux0qs1ZvQ@mail.gmail.com/

The current connectivity check delegates to rev-list, which walks
the full object closure at the connectivity boundary. On a large
private repository with ~600K reachable trees and blobs, fetching
a single new commit takes ~1.8 seconds -- almost all of it spent
walking objects unrelated to the incoming change.

I think this is worth optimizing, since the connectivity check
is on the critical path in normal fetch and receive-pack flows.

The core idea here is to compare each incoming commit's tree
against its parent trees, descending only into entries that
differ. This shifts much of the tree-verification cost toward
changed paths rather than the full reachable closure. On the
same repository, the same fetch drops to ~140 ms (12.9x faster).
On linux.git it drops from 226 ms to 97 ms (2.3x).

Synthetic benchmarks (p5412) show 2.5x-19.3x speedups for
single-commit checks across tree sizes from 50K to 800K
files, with breakeven around 500 commits in the 200K-file
fixture and an observed regression of up to ~1.7x in the long
linear-chain cases tested. The tradeoff comes from rereading
parent trees as comparison bases. (Patch 2 contains the full
table and analysis.)

The series:

1. connected: extract get_self_contained_pack() helper

   Pure refactor for reuse by the incremental path.

2. connected: add incremental connectivity check

   Core implementation, tests, and performance tests. Falls back
   to rev-list for shallow boundaries, partial clones, replacement
   objects, and deepening fetches.

3. connected: enable incremental check with shallow boundaries

   Adds shallow boundary handling and removes the shallow guard.

4. connected: enable incremental check for partial clones

   Adds promisor object handling and removes the promisor guard.
   Missing promisor objects are accepted without triggering lazy
   fetches.

The series includes a test-tool helper (test-check-connected) that
exercises check_connected() directly, making it easier to test
specific object graph shapes and failure modes without going through
the full fetch/push machinery. The functional tests cover happy
paths, corrupt objects, missing objects, shallow boundaries, and
partial clones. There is also a perf test (p5412), though it may
be too specific for this series -- happy to drop it if a more
general regression test would be preferred.

Open questions

Some questions surfaced while building this:

Type confusion: the existing rev-list connectivity check does not
always detect a tree entry whose mode disagrees with the actual
object type. The incremental verifier retains this same
behavior -- it checks existence but not type correctness for
blobs. Should we tighten type checking in a follow-up covering
both the rev-list and incremental paths, or is the current
behavior sufficient as-is?

Replacement objects: the existing rev-list path follows replacement
refs, while git-prune explicitly disables them. The incremental
path falls back to rev-list when replacement objects are active.
Should connectivity checking operate on the underlying object graph
rather than the replaced one? Even the rev-list behavior here may
be worth reconsidering.

The tree verifier introduces private _nofetch variants of tree
reading and tag peeling to avoid triggering lazy promisor fetches
during verification. This felt a bit awkward and introduces some
code duplication, but trying to make the existing functions support
nofetch started to become a can of worms so this at least keeps
the change smaller than it otherwise would have been. Is this
something that should be refactored first or is keeping the
duplication acceptable?

I am not sure how to reason about the regression in some edge-cases.
Perhaps it's not a problem since it's gated behind a config flag,
but that might mean the flag needs to remain permanently?
Alternatively, it would be possible to build some type of heuristic
to decide which algorithm to use based on how many commits need to
be verified, or how many files exist in the repository.
Or is a constant factor regression in some unfavorable cases an
acceptable price to pay for helping the happy cases?

Next steps

I don't want to digress too much here, but I think it's
useful to view this as one part of making connectivity-check
cost less dependent on total repository size. This series
addresses tree verification; the other substantial part is
finding the commit boundary.

In the same single-commit case on my large private repository, a
separate prototype using an in-process boundary walk reduces the
total incremental connectivity check from ~140 ms to ~14 ms.
That work is not included here; boundary discovery has its own
tradeoffs and seems better reviewed separately.

Thanks,
Kristofer