Skip to content

Add partial purge timeout support in gRPC proto mapping#3398

Open
YunchuWang wants to merge 3 commits intodevfrom
wangbill/partial-purge-timeout
Open

Add partial purge timeout support in gRPC proto mapping#3398
YunchuWang wants to merge 3 commits intodevfrom
wangbill/partial-purge-timeout

Conversation

@YunchuWang
Copy link
Copy Markdown
Member

Summary

Map the new timeout proto field in PurgeInstanceFilter to DTFx's PurgeInstanceFilter.Timeout, and forward PurgeResult.IsComplete back to the proto response. This enables the isolated worker SDK's partial purge feature to work end-to-end through the gRPC boundary.

Motivation

The isolated worker SDK communicates with the Functions host via gRPC. When purging large numbers of instances, the gRPC call times out before the host-side purge completes, causing:

  1. OperationCanceledException on the host side
  2. RpcException(StatusCode.Internal) on the worker side
  3. Deleted count = 0 reported (even though rows were actually deleted)

With this change, the SDK can pass a timeout through the proto, and the host maps it to DTFx's PurgeInstanceFilter.Timeout. The purge stops within the timeout, returns accurate counts, and sets isComplete = false so the caller knows to loop.

Changes

  • Proto: Add google.protobuf.Duration timeout = 4 to PurgeInstanceFilter message
  • ProtobufUtils.ToPurgeInstanceFilter: Map proto timeout → DTFx PurgeInstanceFilter.Timeout
  • ProtobufUtils.CreatePurgeInstancesResponse: Map DTFx PurgeResult.IsComplete → proto isComplete (bug fix — was previously not being set despite field existing in proto)

Breaking Changes

None. Additive proto field + bug fix for existing but unpopulated response field.

Dependencies

Benchmark Results (500K instances, EP1, isolated worker)

Metric Without timeout With timeout (25s)
Reported deleted 17,402 (3.5%) 499,560 (99.9%)
Errors 41 (95% failure) 0
Purge rate 12.3 inst/s 318.1 inst/s

Replaces #3397 (which had unrelated agent changes).

Copilot AI review requested due to automatic review settings March 18, 2026 19:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the host-side gRPC/proto mapping to support partial purge semantics by forwarding a purge timeout from the worker to DTFx and returning completion status back across gRPC. It also introduces new GitHub Actions workflows and agent instruction documents (which are not described in the PR summary and materially expand the PR scope).

Changes:

  • Add timeout to PurgeInstanceFilter proto and map it to DTFx PurgeInstanceFilter.Timeout.
  • Forward PurgeResult.IsComplete into the gRPC purge response (isComplete).
  • Add scheduled GitHub Actions workflows plus Copilot/agent instruction documents for automated verification and daily review.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
src/WebJobs.Extensions.DurableTask/ProtobufUtils.cs Maps proto timeout into DTFx purge filter; attempts to map purge completion into proto response.
src/WebJobs.Extensions.DurableTask/Grpc/Protos/orchestrator_service.proto Adds google.protobuf.Duration timeout = 4 to PurgeInstanceFilter.
.github/workflows/pr-verification.yaml Adds a scheduled PR verification agent workflow (Azurtite + Copilot CLI + label/comment automation).
.github/workflows/daily-code-review.yaml Adds a scheduled daily code review agent workflow (build/test + Copilot CLI + PR/issue automation).
.github/copilot-instructions.md Adds general AI assistant architectural guidance for the repo.
.github/agents/pr-verification.agent.md Adds detailed instructions for the PR verification agent behavior.
.github/agents/issue-triage.agent.md Adds detailed instructions for an issue triage agent behavior.
.github/agents/daily-code-review.agent.md Adds detailed instructions for a daily code review agent behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/WebJobs.Extensions.DurableTask/ProtobufUtils.cs
Comment thread .github/workflows/pr-verification.yaml Outdated
Comment thread .github/workflows/pr-verification.yaml Outdated
Comment thread .github/workflows/pr-verification.yaml Outdated
Comment thread .github/workflows/daily-code-review.yaml Outdated
Comment thread src/WebJobs.Extensions.DurableTask/ProtobufUtils.cs
Comment thread .github/workflows/pr-verification.yaml Outdated
Comment thread .github/workflows/daily-code-review.yaml Outdated
Comment thread .github/workflows/pr-verification.yaml Outdated
@YunchuWang YunchuWang force-pushed the wangbill/partial-purge-timeout branch 2 times, most recently from e3a319e to 748c68a Compare March 19, 2026 01:48
Copilot AI review requested due to automatic review settings March 19, 2026 01:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the gRPC ↔ DTFx mapping layer in WebJobs.Extensions.DurableTask to better support partial purge scenarios by propagating additional purge result metadata back to gRPC callers.

Changes:

  • Propagates PurgeResult.IsComplete into the gRPC PurgeInstancesResponse.isComplete field.
  • Minor formatting/whitespace adjustments in ProtobufUtils event mapping blocks.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/WebJobs.Extensions.DurableTask/ProtobufUtils.cs
Comment thread src/WebJobs.Extensions.DurableTask/ProtobufUtils.cs
- Add timeout field (4) to PurgeInstanceFilter proto message
- Map proto timeout to DTFx PurgeInstanceFilter.Timeout
- Forward PurgeResult.IsComplete to proto response (bug fix)
- Zero breaking changes
- Tests for CreatePurgeInstancesResponse: IsComplete true/false/null mapping
- Tests for ToPurgeInstanceFilter: basic fields, status filter, empty status list
- Timeout mapping test deferred until DTFx Core dependency includes PurgeInstanceFilter.Timeout
Copilot AI review requested due to automatic review settings March 19, 2026 21:42
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the gRPC↔DTFx purge mapping to support partial purges by flowing a new timeout field from PurgeInstanceFilter into DurableTask.Core.PurgeInstanceFilter.Timeout, and by returning PurgeResult.IsComplete back to the gRPC response so callers can loop until completion.

Changes:

  • Add google.protobuf.Duration timeout to the PurgeInstanceFilter proto message.
  • Map proto timeout → DTFx PurgeInstanceFilter.Timeout in ProtobufUtils.ToPurgeInstanceFilter.
  • Populate PurgeInstancesResponse.isComplete from PurgeResult.IsComplete, and add unit tests for isComplete mapping + basic purge-filter mapping.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
src/WebJobs.Extensions.DurableTask/Grpc/Protos/orchestrator_service.proto Adds the new timeout field to the purge filter message.
src/WebJobs.Extensions.DurableTask/ProtobufUtils.cs Implements request/response mapping for purge timeout and isComplete.
test/FunctionsV2/ProtobufUtilsTests.cs Adds tests for isComplete propagation and basic purge filter conversions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/FunctionsV2/Tests/Unit/ProtobufUtilsTests.cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants