ADR-0037 differentiable SDF ray intersection, rebased onto main - #3
Merged
Conversation
…policy, PTX source identity, scene_multipath split)
Phase 0 of docs/dev/sdf_intersection_plan.md. The record pins the parts of the primitive that are expensive to change once a downstream depends on them, before any code exists: - vertex-centered dense grid, trilinear interpolation, negative-inside sign convention, world-metric caller-baked distances; - oriented-box placement with scalar-first quaternion and full side lengths, the rigid world-to-local map, and the grid coordinate mapping; - slab-clipped sphere trace with entry-sign-frozen relaxed stepping and a bounded sign-flip bisection fallback that always reports a proven crossing; - frozen-winner IFT derivatives for all six gradient inputs, with the closed form of every partial, the differentiably recomputed normal, the grazing denominator clamp, and the atomics determinism statement; - device-derived default eps_hit so the operation never reads a device tensor on the host, and reuse of the existing epsilon and miss sentinel constants; - non-goals, the Phase 4 contract-file impact list, and the Phase 5 backlog. The impact list also records that adding the capability makes the two _capabilities.py copies diverge on four lines rather than the three ADR-0036 enumerates, so that prose is amended in the same change instead of rotting. tests/test_adr0037_sdf_intersection.py guards what exists now: the record's structure, its documented constants against shared/contracts/operations.json, and its agreement with the plan. Its contract and compile-policy assertions are written to stay exact after Phase 3a/4 land rather than needing a rewrite.
Two audit findings on the Phase 0 record.
The pinned termination order tested the sign flip at t_{k+1} before the
t_{k+1} > t_hi exit rule, so the step was sampled beyond the traced interval,
where section 1's trilinear formula linearly extrapolates. A ray clipping a box
corner with a large entry distance could flip sign on that extrapolation and
report a hit at t* > t_hi, past tmax and on geometry the grid does not
represent. The step is now clamped, t_k+1 = min(t_raw_k, t_hi), the exit rule
fires on the unclamped t_raw_k, and section 1 clamps the grid coordinate to
[0, N_i - 1] so the interpolant has no extrapolated branch at all. Every
reported t* now satisfies t_lo <= t* <= t_hi <= tmax by construction. The guard
test pins the clamp and the interval bound.
The compile-policy guard was written against a nonexistent schema:
translation_units is a list of {backend, unit, source, profile} entries, not a
backend-keyed mapping of object lists, so the assertion body would have raised
AttributeError the moment Phase 3a added an SDF unit instead of checking its
profile. It now iterates the list directly; verified by injecting a no_fmad SDF
entry and confirming the assertion fires.
Also adds the ADR-0036 forward reference to ADR-0037's pending four-line
_capabilities.py amendment, so a Phase 4 implementer reading only ADR-0036 sees
it.
ADR-0037 phase 1. `_sdf_reference.py` is the test-only numerical oracle for the Phase 3 CUDA kernels: a fully detached sphere trace with entry-sign relaxation, a step clamped to `t_hi` before it is sampled, sign-flip bisection with the section 4 rule order, and a differentiable last step implementing the frozen-winner implicit function theorem with the signed grazing clamp. The trilinear gather is written out by hand rather than delegated to `grid_sample` because ADR-0037 clamps `u` before the base/fraction split, freezes the base voxel index on the tape, and needs the analytic `dD/du` in closed form; the module docstring records that choice and a test pins the equivalence. `test_sdf_reference.py` checks the reference against sources independent of it: closed-form ray/sphere and ray/box hits of the baked analytic fields, exact recovery on a linear field, inside-start rays, bitwise-inert misses, an OBB clip that evaluates no field sample when it rejects a lane, non-eikonal recovery through bisection, `grid_sample(align_corners=True)` for the interpolant, `gradcheck` on the frozen tape, central finite differences of a re-marched forward pass for all six gradient inputs, a grazing sweep through tangency, and a constant field that forces the `sign(0) := +1` clamp branch.
Phase 2 of docs/dev/sdf_intersection_plan.md. Two new backend-neutral headers own the ADR-0037 numerics that the Torch kernels will instantiate in Phase 3: - shared/sdf/grid_sdf.cuh: vertex-centred trilinear sampling of a dense signed distance grid, with the corner indices and weights a values gradient scatters through and the analytic index-space gradient. - shared/sdf/sphere_trace.h: the oriented-box placement and slab clip, the contract epsilons, the on-device eps_hit derivation, and the sampler-templated relaxed march with sign-flip bisection. Both are host-compilable, spell only RAYD_HOST_DEVICE, and stay outside every committed-PTX include closure, so the committed Dr.Jit PTX headers are untouched. No existing shared header changes. tests/native/sdf_shared_math_smoke.cpp is compiled with the host compiler and then run, checking the interpolant, the clip, the placement transform, the entry-sign rule, the bisection fallback and the miss cases against closed-form answers; tests/test_sdf_shared_math.py drives that gate and pins the header contract. Verified against the Phase 1 pure-PyTorch reference on twelve rays (eikonal and non-eikonal fields, rotated and translated placements, inside starts, misses, tmax clipping): hit/miss, hit distance and field evaluation counts agree, and the headers also compile as device code with nvcc.
Phase 3a of docs/dev/sdf_intersection_plan.md: the ADR-0037 SDF intersection primitive as three CUDA operations on the Torch backend. The primal marches the shared sphere-trace core over the shared trilinear grid sampler and writes the public result plus the frozen-winner tape (hit distance, hit mask, base voxel index). The derivative translation unit carries both modes of the frozen-winner implicit function theorem: neither re-marches, both rebuild the interpolant on the taped voxel, and they are exact duals over the six supported inputs. The values gradient scatters through the eight trilinear weights with atomics; position, rotation and scale accumulate per grid; origins and directions are per-ray and exact. Missed lanes are bitwise inert: every lane writes its whole row once in the primal, and a missed lane contributes no atomic and no nonzero derivative. The grazing denominator carries the ADR-0037 magnitude floor with sign(0) := +1, so no NaN or infinity other than the `t` miss sentinel can leave the operation. The three ops are registered GIL-free from the start through pure C++ implementations. Validation is structural only, so the operation performs no device-to-host copy, no stream synchronization and no host read of a device tensor; a non-positive eps_hit stays the device-derivation sentinel. Both new CUDA translation units take the nvcc_default profile with no per-source flag block, and enter shared/contracts/compile_policy.json in the same change so the flag contract stays green rather than being repaired later. No shared header is modified and no committed-PTX include closure is entered, so the PTX source digest is unaffected. The Torch ABI audit is regenerated for the three new dispatcher operators.
Phase 3b of docs/dev/sdf_intersection_plan.md: the public Torch surface over the ADR-0037 native operations. `SdfGrid` is a frozen holder that validates structure only, so construction never reads a device value and never synchronizes; the value conditions the ADR leaves to the device path stay there. `sdf_intersect` validates the ray batch and the four host march parameters, then dispatches: a request with no reverse or forward gradient goes straight to the native forward operation, and anything else goes through `_SdfIntersectFunction`. `eps_hit=None` is translated to the non-positive device-derivation sentinel rather than to a host computation of `h_min`. The autograd Function keeps the six differentiable inputs first so `ctx.needs_input_grad` maps onto the native gradient request one to one, saves the frozen tape for both modes, and marks the tape and the mask and step counters non-differentiable. Backward and JVP consume that tape and never re-march. Upstream gradients and tangents are made contiguous before they cross the boundary because autograd hands out an expanded stride-0 gradient for the common `output.sum()` objective, which the native contiguous row arithmetic cannot read. The tests run on the GPU against the Phase 1 reference: forward parity including the step count, all six gradients, and forward-mode parity against the reference with an explicitly frozen tape. Running the whole reference under a dual level instead would let tangents leak through the march, which is not the frozen-winner contract, so the JVP is also pinned by exact duality with the backward pass. Also covered: float32 gradcheck on a linear field, misses from every rejection path being bitwise inert in value and in gradient, a grazing sweep leaving no NaN and no infinity other than the `t` miss sentinel, forward determinism across repeats and across the two dispatch paths, and rejection of a host, double, strided, misshapen, or mismatched input with an actionable message. No native code changes, so no rebuild is involved.
Phase 4 of docs/dev/sdf_intersection_plan.md lands the contract surface for the ADR-0037 differentiable SDF ray intersection as one change. - public_api.json gains the `sdf_intersect` capability key, its core/provisional API entry, and both backends' booleans (drjit false, torch true). - operations.json gains the key in `required_capability_keys`, the full `operations.sdf_intersect` entry with inputs, representation, ray domain, fixed-winner AD, and the numeric policy including the grazing clamp and the `inf` miss sentinel, plus the `sdf_intersection` result contract with its miss block. - Both `_capabilities.py` copies gain the key in the same change and repin `_SCHEMA_SHA256`; ADR-0036 is amended from three divergent lines to four, as ADR-0037's contract impact list requires. - The hard-coded operation set and integration matrix in tests/test_shared_operation_contract.py, and the Torch stub module list in tests/test_public_api_manifest.py, are updated to match. - tests/test_adr0037_sdf_intersection.py drops its pre-Phase-4 skips and now cross-checks the record against the declared operation, the shared epsilons, the compiled device constants, the Python defaults, and both capability copies. - README.md and backends/torch/README.md record the user-visible surface. The new CUDA translation units were already declared under `nvcc_default` in compile_policy.json by Phase 3a; the ADR guard now asserts that profile instead of skipping.
The SDF work was developed on a line whose Torch CMakeLists never touched CMAKE_CUDA_FLAGS, so its ADR-0035 gate banned that identifier outright. Main writes it to strip the per-architecture `-gencode` pairs Caffe2 leaves behind, which nvcc would otherwise honour alongside the grouped families and compile every architecture twice. gencode is not one of the four numeric profiles, so the assertion is tightened rather than relaxed: the four string() statements are pinned, each must stay numeric-flag-free, the only set() naming the variable is the caller-value capture, and add_compile_options() may not name it at all. Regenerates abi_audit.json. Its source digest covers the Torch CMakeLists, which the rebase replaced with main's, plus library.cpp and _stable.py, which the SDF commits extended with the three sdf_intersect operators.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the SDF intersection line onto
mainwithout the CI regression it was stacked on.Why this is a rebase, not a merge
The SDF work was developed on a line that diverged from
mainate892935. Its first commit,4f0e953, deletedci.ymland the CI acceleration thatmainstill carries (grouped gencode,RAYD_NVCC_LAUNCHER, sccache, opt-in triggers). Merging that line would have reverted all of it.This branch replays the nine feature commits onto
origin/mainwith--onto, dropping4f0e953.wt/sdf-intersectionis untouched.Contents
Two reconciliation fixes
ADR-0035 gate. The SDF line's contract test banned the
CMAKE_CUDA_FLAGSidentifier outright, because its Torch CMakeLists never wrote it. Main writes it to strip the per-architecture-gencodepairs Caffe2 leaves behind; nvcc honours both spellings at once and would compile every architecture twice. gencode is not one of the four numeric profiles the ADR governs, so the assertion is tightened rather than relaxed: the fourstring()statements are pinned, each must stay numeric-flag-free, the onlyset()naming the variable is the caller-value capture, andadd_compile_options()may not name it.ABI audit.
abi_audit.jsonis regenerated. Its source digest covers the TorchCMakeLists.txt, which the rebase replaced with main's, pluslibrary.cppand_stable.py, which the SDF commits extended with the threesdf_intersectoperators.Verification
319 passed, 13 skipped, 566 subtests, on
witwin3. The three packaging-metadata modules that gate the release workflow pass individually.Native compilation is not covered here; it needs a hosted run.
🤖 Generated with Claude Code