Where it lives: main. Found while working on feature/granite-turboctc-default (ef43e69).
GraniteSwitchForCausalLM lists HasInnerState among its bases and implements none of it. Same inherited-marker mistake as IsHybrid (dropped on the TurboCTC branch): it arrived from GraniteMoeHybridConfig rather than by design.
Unlike IsHybrid, this one is not merely cosmetic. On vLLM 0.20.2:
# vllm/config/vllm.py:1797-1802
def _validate_v2_model_runner(self) -> None:
"""Check for features not yet supported by the V2 model runner."""
unsupported: list[str] = []
if self.model_config is not None and self.model_config.has_inner_state:
unsupported.append("hybrid/mamba models")
So declaring it excludes the model from vLLM's V2 model runner — we are opted out of a code path on the strength of a claim that is not true. The check is gone from that file in 0.26.0+, so the cost is specific to the version line we are on.
Fix: drop the declaration, after checking nothing else reads has_inner_state for us. Plausibly a free win rather than a cost. Should be measured (does the V2 runner actually engage, and does anything change?) rather than assumed.
Where it lives:
main. Found while working onfeature/granite-turboctc-default(ef43e69).GraniteSwitchForCausalLMlistsHasInnerStateamong its bases and implements none of it. Same inherited-marker mistake asIsHybrid(dropped on the TurboCTC branch): it arrived fromGraniteMoeHybridConfigrather than by design.Unlike
IsHybrid, this one is not merely cosmetic. On vLLM 0.20.2:So declaring it excludes the model from vLLM's V2 model runner — we are opted out of a code path on the strength of a claim that is not true. The check is gone from that file in 0.26.0+, so the cost is specific to the version line we are on.
Fix: drop the declaration, after checking nothing else reads
has_inner_statefor us. Plausibly a free win rather than a cost. Should be measured (does the V2 runner actually engage, and does anything change?) rather than assumed.