When a URL waits in the queue longer than fetcher.timeout.queue, FetcherThread just logs and moves on: no status, no metric, tuple acked. The comment assumes the tuple already failed at Storm level, but that's only true if topology.message.timeout.secs is shorter — otherwise the URL vanishes and the status index never hears about it.
Emit FETCH_ERROR? At least a metric?
Noticed while planning the characterization tests for #2057 — those will pin the current behaviour for now, the fix would be its own PR.
When a URL waits in the queue longer than fetcher.timeout.queue, FetcherThread just logs and moves on: no status, no metric, tuple acked. The comment assumes the tuple already failed at Storm level, but that's only true if topology.message.timeout.secs is shorter — otherwise the URL vanishes and the status index never hears about it.
Emit FETCH_ERROR? At least a metric?
Noticed while planning the characterization tests for #2057 — those will pin the current behaviour for now, the fix would be its own PR.