Summary
Follow-up to #6461 (closed, documented in NEWS.md via #6572). That issue covered the case where the alloy migration turns a previously-masked subgraph bug into a visible deterministic failure. We have confirmed a second failure mode: when a mapping calls ethereum.decode() with a type string that ethabi accepted but alloy rejects (here: the invalid type bytes128) and handles the null result with an early return, the subgraph keeps indexing and reports healthy/synced while silently dropping entities. There is no fatal error, no unhealthy status, and _meta.block stays at chain head, so the data loss is not surfaced to the indexer or to consumers.
Affected deployments
Circle CCTP USDC MessageTransmitter subgraphs:
| Deployment |
Network |
Contract |
QmWgi6hNfwCGiTAhH7gTSMSfvvYUPRbBQSjRmvuviRGGwy |
mainnet |
0x0a992d191deec32afe36203ad87d7d289a738f81 |
QmQtNd36amtQ8h8GF5rwkLLWyyBGwqad3j3WgZAMuLvDMd |
arbitrum-one |
L2 MessageTransmitter |
Both carry active curation signal and are served through the gateway.
Root cause
The subgraph source is OffchainLabs/arbitrum-subgraphs, packages/cctp/src/usdc-message-transmitter.ts. handleMessageSent(MessageSent(bytes)) re-pads the packed CCTP message and calls:
ethereum.decode("(uint32,uint32,uint32,uint64,bytes32,bytes32,bytes32,bytes128)", messagePadded)
bytes128 is not a valid ABI type (fixed-size bytes stop at bytes32). The ethabi type parser (≤ 0.41) accepted it anyway as FixedBytes(128); the alloy parser (≥ 0.42, PR #6063) correctly rejects the type string — same bug class as the leading-space " address" case in #6461. decode returns null and the mapping logs and returns:
ERRO decodedMessageData doesn't exist, data_source: L1USDCMessageTransmitter, ... component: SubgraphInstanceManager > UserMapping
MessageReceived events on the same contract decode canonically and keep working, so the deployment looks fully healthy.
Evidence
- Blocks containing MessageSent events reach the runner (
Start processing block, triggers: N) but commit entities: 0; the ERRO line above is logged continuously.
- Confirmed by disassembling the deployed WASM (
wasm2wat): the decode call sits behind an if null → log + return on both decode paths (decodedMessageData, messageBodyData).
- On our indexer,
messageSent entity creation stops on both chains within 5 minutes of our v0.41.1 → v0.42.1 rollout (2026-06-12 ~12:20 UTC), while messageReceived continues to head.
Network impact
Each indexer starts losing messageSent entities from its own upgrade date onward. We confirmed the graph-node version of every indexer allocated on these deployments via their public status endpoint ({ version { version } }), and the correlation with dataset correctness is exact:
| Indexer |
graph-node |
messageSent dataset |
| Lunanova |
0.36.0 |
correct |
| decisionbasis |
0.41.2 |
correct |
| ellipfra (us) |
0.42.1 |
stops 2026-06-12 (our upgrade) |
| Pinax (riv-dev) |
0.44.0 |
stops 2026-06-24 (their upgrade) |
Consequences observed in production (reported to us by a consumer):
- The gateway serves stale/incomplete data for the majority of queries (most servable indexers are on ≥ 0.42).
- POIs diverge indexer-by-indexer with no failure anywhere; consistency checks between two ≥ 0.42 indexers pass while both are wrong.
- As the network converges on ≥ 0.42, correct copies disappear entirely. Rewind/resync on ≥ 0.42 reproduces the same gaps, so the deployments are unrecoverable on current graph-node; only a republished subgraph (parsing the message with byte slices) fixes it going forward, ideally grafted below the oldest 0.42-upgrade block or resynced from scratch.
Suggestion
We understand from #6461 that no backwards-compatible decode fix is planned, and we agree the mapping is at fault. However, given the silent failure mode, would you consider:
- Logging decode failures at a level/shape that operators can alert on (the current ERRO comes from the user mapping only — a mapping that returns without logging is fully invisible), e.g. a host-side counter/metric
ethereum_decode_failures{deployment};
- Mentioning the silent-loss mode explicitly in the NEWS.md entry for 0.42, so indexers know that "healthy + synced" does not rule out this class of data loss.
Happy to provide full logs, WASM disassembly, and per-indexer dataset comparisons.
Summary
Follow-up to #6461 (closed, documented in NEWS.md via #6572). That issue covered the case where the alloy migration turns a previously-masked subgraph bug into a visible deterministic failure. We have confirmed a second failure mode: when a mapping calls
ethereum.decode()with a type string that ethabi accepted but alloy rejects (here: the invalid typebytes128) and handles thenullresult with an early return, the subgraph keeps indexing and reports healthy/synced while silently dropping entities. There is no fatal error, no unhealthy status, and_meta.blockstays at chain head, so the data loss is not surfaced to the indexer or to consumers.Affected deployments
Circle CCTP USDC MessageTransmitter subgraphs:
QmWgi6hNfwCGiTAhH7gTSMSfvvYUPRbBQSjRmvuviRGGwy0x0a992d191deec32afe36203ad87d7d289a738f81QmQtNd36amtQ8h8GF5rwkLLWyyBGwqad3j3WgZAMuLvDMdBoth carry active curation signal and are served through the gateway.
Root cause
The subgraph source is
OffchainLabs/arbitrum-subgraphs,packages/cctp/src/usdc-message-transmitter.ts.handleMessageSent(MessageSent(bytes))re-pads the packed CCTP message and calls:bytes128is not a valid ABI type (fixed-size bytes stop atbytes32). The ethabi type parser (≤ 0.41) accepted it anyway asFixedBytes(128); the alloy parser (≥ 0.42, PR #6063) correctly rejects the type string — same bug class as the leading-space" address"case in #6461.decodereturns null and the mapping logs and returns:MessageReceivedevents on the same contract decode canonically and keep working, so the deployment looks fully healthy.Evidence
Start processing block, triggers: N) but commitentities: 0; the ERRO line above is logged continuously.wasm2wat): the decode call sits behind anif null → log + returnon both decode paths (decodedMessageData,messageBodyData).messageSententity creation stops on both chains within 5 minutes of our v0.41.1 → v0.42.1 rollout (2026-06-12 ~12:20 UTC), whilemessageReceivedcontinues to head.Network impact
Each indexer starts losing
messageSententities from its own upgrade date onward. We confirmed the graph-node version of every indexer allocated on these deployments via their public status endpoint ({ version { version } }), and the correlation with dataset correctness is exact:Consequences observed in production (reported to us by a consumer):
Suggestion
We understand from #6461 that no backwards-compatible decode fix is planned, and we agree the mapping is at fault. However, given the silent failure mode, would you consider:
ethereum_decode_failures{deployment};Happy to provide full logs, WASM disassembly, and per-indexer dataset comparisons.