Preserve TypeVar narrowing across concrete dispatch calls#3610
Open
mikeleppane wants to merge 1 commit into
Open
Preserve TypeVar narrowing across concrete dispatch calls#3610mikeleppane wants to merge 1 commit into
mikeleppane wants to merge 1 commit into
Conversation
A function parameterised over a TypeVar bound to a union (e.g. `T: A | B`) commonly dispatches on the concrete member with `isinstance`/`type(x) is C` and returns the result of a helper typed for that member. Narrowing already produces the intersection `A & T` for the value, but passing it through a concrete helper such as `def func_a(a: A) -> A` collapsed the result back to `A` at the call boundary, so the declared `-> T` return failed with a spurious bad-return. Pyright accepts this dispatch idiom; Pyrefly did not. Refine a call result back to the narrowed witness when the argument provably *is* that witness: the return-context hint is a bounded TypeVar, the helper's parameter type equals its return type, and the argument's type is an intersection carrying both that TypeVar and the concrete return type. In that case the bare result is upgraded back to `Concrete & T`, which is assignable to `T`. The parameter==return and witness checks are the soundness gates: a fresh concrete value (e.g. `func_a(A())`) or a helper returning a subtype is never the witness and stays correctly rejected. The refinement accepts a single positional or a single keyword argument (both bind one value to the one parameter) and applies to any bounded TypeVar, not only union bounds — the soundness gates do not depend on the bound's shape. Fixes facebook#3550
|
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
Member
This was my first thought when I saw the initial example. A function I'm not convinced we should follow Pyright's lead on this example. Edit to add: there is already a way to describe a function that returns its argument, generics. Making |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
What
Pyrefly rejected a dispatch-on-concrete-subclass pattern that pyright accepts. A function over a
TypeVarbound to a union, narrowing withisinstance/type(x) is Cand returning a helper's result, got a spuriousbad-return.Fix in
pyrefly/lib/alt/call.rs—refine_typevar_bound_call_result. Plus two scope expansions over the base fix: keyword args + any bounded TypeVar (not just union bounds). Tests inpyrefly/lib/test/narrow.rs.Why
Type theory: inside the body
Tis rigid (skolemized), constraintT <: A | B.T <: A— using aTasA= fine (down).A <: T— false in general: caller'sTcould be a strict subtypeA2 <: A. Can't fabricate aTfrom a bareA(up = unsound).Narrowing already does the right thing:
isinstance(x, A)gives the meetA & T(reveal_type(x)=A & T), which is both<: A(passable tofunc_a) and<: T(returnable).The leak is at the call boundary.
func_a: (A) -> Ais monomorphic — applying it computes codomainAand drops the& T. Sofunc_a(A & T)typed as plainA→ not assignable to-> T→ spurious error. Narrowing the variable can't fix it; the value comes from the helper's signature, not fromx.How
After computing a call's result
ty, refine it back to the witness — but only when the argument provably is that witness. Gates, in order (cheap → expensive):TypeVar(Restriction::Bound(_))expected_types[arg_range] == ty— helper's param type == return type (the identity shape; reuses already-computed data, previously discarded as_expected_types)Intersectcarrying both that same skolemTand the return typety→ return the intersection (A & T) instead of bareAA & T <: T→ checks. Expensiveexpr_infergated behind cheap checks.Soundness = gates 3+4. A function with
param == returnbehaves (parametrically) like it returns its input; when the input carries& T, the output's precise type still carries& T— we recover the witness, not invent it. Anything else (freshA, helper returning a subtype(A)->A3) fails a gate → stays rejected.Honest residual:
def f(a: A) -> A: return A()(returns fresh, not input) is accepted though mildly unsound — inherent (the signature can't say "returns its arg"), matches pyright.Why these two expansions are safe: keyword
f(a=x)≡ positionalf(x)(one value, one param) — zero new soundness surface.Bound(Union)→Bound(_)— gates 3+4 don't depend on the bound's shape, so per-instance soundness is identical; only applicability widens. Constrained TypeVars (TypeVar("T", A, B)) already worked soundly via exact-identity and need no special-casing.Test plan
5 testcases in
narrow.rs:test_typevar_dispatch_on_concrete_classtype(x) is A+isinstancedispatch OK;func_a(A())fresh →bad-returntest_typevar_dispatch_keyword_argfunc_a(a=x)OKtest_typevar_dispatch_single_boundT: POK;reveal_type(x)=C1 & Tbtest_typevar_dispatch_negatives(A)->A3subtype-return → rejected;func_a(a=A())fresh keyword → rejectedVerification:
cargo test -p pyrefly 'test::narrow::'→ 199 passed, 0 failedpython3 test.py→ formatting ✅, lint ✅, 5286 lib tests ✅, conformance ✅. (jsonschema phase couldn't run locally — missingjsonschema/tomlpip deps; env gap, not code.)mypy_primer×3 runs, ~17 ecosystem projects (pydantic, pandera, sphinx, aiohttp, starlette, tornado, werkzeug, mypy, attrs, sympy, trio, jinja, packaging, rich…) → zero diagnostic diffsFixes #3550