Skip to content

core: Reset configSelector on realChannel while entering IDLE state - #12832

Merged
kannanjgithub merged 3 commits into
grpc:masterfrom
kkalin68:xds-fix-channel-idle-limbo
Sep 4, 2026
Merged

core: Reset configSelector on realChannel while entering IDLE state#12832
kannanjgithub merged 3 commits into
grpc:masterfrom
kkalin68:xds-fix-channel-idle-limbo

Conversation

@kkalin68

@kkalin68 kkalin68 commented May 27, 2026

Copy link
Copy Markdown
Contributor

ManagedChannel will be stuck in IDLE state when xDS control plane doesn't have a resource anymore.
The scenario is following:

  1. Channel is open for a xds resource.
  2. XdsNameResolver subscribes to the resource on xDS control plane.
  3. The resource is removed from xDS control plane for extended period of time (unhealthy for more than the idle timeout on Channel)
  4. Channel enters into TRANSIENT_FAILURE
  5. The Idle timeout triggers and Channel shutdowns XdsNameResolver and other resources. xDS watchers are removed.
  6. The resource comes back online on xDS control plane.
  7. A new GRPC call is executed targeting the channel.
  8. The channel stays in IDLE state and reports: "io.grpc.StatusRuntimeException: UNAVAILABLE: LDS resource xxxx does not exist nodeID: yyyy" because realChannel.configSelector still points to old state.
public <ReqT, RespT> ClientCall<ReqT, RespT> newCall(
        MethodDescriptor<ReqT, RespT> method, CallOptions callOptions) {
      if (configSelector.get() != INITIAL_PENDING_SELECTOR) {
        return newClientCall(method, callOptions);
      }
...

@linux-foundation-easycla

linux-foundation-easycla Bot commented May 27, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: kkalin68 / name: Konstantin Kalin (f09b7d9)

@AgraVator

AgraVator commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Hey,
Please fix the failing test cases. Thanks.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a ManagedChannel idle-mode edge case where a stale InternalConfigSelector can prevent the channel from properly exiting IDLE after xDS resources disappear and later reappear.

Changes:

  • Reset realChannel’s config selector to INITIAL_PENDING_SELECTOR when entering IDLE, forcing the next call to trigger resolver/LB restart.
  • Prevent pending-call reprocessing when updateConfigSelector() is called with INITIAL_PENDING_SELECTOR (only reprocess when transitioning away from the initial pending selector).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread core/src/main/java/io/grpc/internal/ManagedChannelImpl.java
@kkalin68

kkalin68 commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Hey, Please fix the failing test cases. Thanks.

The tests are passing locally. Also the failures in tests(11) and tests(17) runs are unrelated to changes I made. And same tests passed in tests(8) and test(21). Also I don't see an option to retrigger the failed tests.

@kkalin68

kkalin68 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Is it anything else I should do to get the change reviewed ? We face this issue occasionally in production and it requires a service restart to reset GRPC channel

@kannanjgithub

Copy link
Copy Markdown
Contributor

Is it anything else I should do to get the change reviewed ? We face this issue occasionally in production and it requires a service restart to reset GRPC channel

Can you address the Copilot review comment above for adding a new test?

@kkalin68

Copy link
Copy Markdown
Contributor Author

Is it anything else I should do to get the change reviewed ? We face this issue occasionally in production and it requires a service restart to reset GRPC channel

Can you address the Copilot review comment above for adding a new test?

I added tests. Please note, I used Copilot to navigate and create the tests

@kkalin68

Copy link
Copy Markdown
Contributor Author

Failed test(11) is not related to my changes. Same test passed for other Java versions

@kkalin68

Copy link
Copy Markdown
Contributor Author

Is it anything else I should do to get the change reviewed ? We face this issue occasionally in production and it requires a service restart to reset GRPC channel

Can you address the Copilot review comment above for adding a new test?

Is there anything I should do to get this merged ?

@kannanjgithub kannanjgithub added the kokoro:force-run Add this label to a PR to tell Kokoro to re-run all tests. Not generally necessary label Aug 18, 2026
@grpc-kokoro grpc-kokoro removed the kokoro:force-run Add this label to a PR to tell Kokoro to re-run all tests. Not generally necessary label Aug 18, 2026
@kkalin68

kkalin68 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

@AgraVator Could you please take a look at the change ?

@AgraVator

Copy link
Copy Markdown
Contributor

@AgraVator Could you please take a look at the change ?

Hey @kkalin68 feel free to merge as @kannanjgithub has already approved the changes

@kkalin68

kkalin68 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

@AgraVator @kannanjgithub GH tells me I cannot merge since PR needs 2 reviewers and only one is approved. Or may be the issue is in failed build for Java 11.

@kannanjgithub kannanjgithub changed the title Reset configSelector on realChannel while entering IDLE state core: Reset configSelector on realChannel while entering IDLE state Sep 4, 2026
@kannanjgithub
kannanjgithub merged commit fc43144 into grpc:master Sep 4, 2026
16 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants