vLLM <=0.25 cannot load a config written by transformers >=5.16 (KeyError: 'full_attention')

[aviv1ron1]

Where it lives: upstream vLLM, reachable only in a venv that pairs transformers >=5.16 with vLLM <=0.25 — i.e. feature/granite-turboctc-default (ef43e69), not main, which pins transformers<5.10.

transformers 5.16 renamed layer type attention -> full_attention and rewrites the value inside PreTrainedConfig.__init__, so even an old config file loads back with the new name. vLLM's table has only the old key:

# vllm/model_executor/models/granitemoehybrid.py:319  (0.19.1 .. 0.25.0)
ALL_DECODER_LAYER_TYPES = {
    "attention": GraniteMoeHybridAttentionDecoderLayer,
    "mamba": GraniteMoeHybridMambaDecoderLayer,
}
...
layer_class = ALL_DECODER_LAYER_TYPES[config.layer_types[layer_idx]]

(EngineCore pid=9824) KeyError: 'full_attention'

Confirmed on both 0.19.1 and 0.20.2. Fixed upstream in vLLM 0.26.0 (0.25.0 still broken), which added "full_attention" alongside.

Two distinct consequences

1. The equivalence tier cannot run — 14 failures + 19 nested. Those tests build their upstream reference with transformers, save_pretrained() it, and hand the directory to vLLM; the saved config now says full_attention and its architectures field routes to vLLM's stale class. So the control group will not start. Our own GraniteSwitchForCausalLM is unaffected: it builds layers through a closure and never consults that table.

Affected: test_skinning_equivalence_vllm[_thorough], test_generation_equivalence, test_granite4_fullsize, test_granite4_mini (9), test_upstream_equivalence.

2. A narrow serving regression. In a venv built from this branch (transformers>=5.16 via the audio extra, vllm<=0.25):

vllm serve ./granite-switch-4.2-3b-audio   # fine - our class, no table
vllm serve ibm-granite/granite-4.0-micro   # KeyError: 'full_attention'

Composed Granite Switch checkpoints are never affected — verified on the published 4.2-30b, whose config carries layer_types: ["full_attention", ...] and serves correctly. Stock Granite 4.0 hybrid-typed checkpoints (granite-4.0-micro, h-small, h-tiny) are. 4.1/4.2 bases are model_type: granite and route to granite.py, which has no such table.

Options

• 4-line alias in our register() — ALL_DECODER_LAYER_TYPES.setdefault("full_attention", ...["attention"]). It has to go here: the crash is in vLLM's spawned engine process, and register() is our vllm.general_plugins entry point, which vLLM loads inside that process (v1/engine/core.py:105). setdefault makes it a no-op on 0.26+. Repairs the venv, not just the tests.
• Test-only sitecustomize.py on PYTHONPATH — keeps src/ free of upstream patches, at the cost of CI plumbing.
• xfail below 0.26 — silences the tests, leaves the serving regression.
• Add a vLLM 0.26+ leg — the real fix. Every module and symbol we import checked clean from 0.20.2 through 0.28.0.

Needs a decision before the TurboCTC branch merges. Related: #12, #14.

Note a separate stale comparison in vllm/config/model.py's is_hybrid escape hatch (layer == "attention") is still broken in 0.28.0. We sidestep it by not declaring IsHybrid; it is worth an upstream PR on its own.