Skip to content

Add async Always Encrypted key cache (1/5) - #5077

Draft
cheenamalhotra wants to merge 3 commits into
mainfrom
dev/cheena/async-ae-key-cache
Draft

cheenamalhotra wants to merge 3 commits into
mainfrom
dev/cheena/async-ae-key-cache

Conversation

@cheenamalhotra

@cheenamalhotra cheenamalhotra commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Description

Layer 1/5 of the split of #4585: async symmetric-key cache foundation only. Adds per-key load gates, cancellable waits, cache-disabled concurrency, and shared sync/async publication. Provider invocation remains serialized without holding the gate across outstanding async I/O.

Extracted from source head 5f1b860410426f8c1af3eb55bfac0e7354bd2ac7 onto main foundation 2bda3c9edb53ce59589cdd401f10ae4415f331a5, preserving newer main changes. Source deviations: explanatory synchronization comments and a self-review fix to recheck the cache after acquiring the provider gate. That avoids redundant decryption and possible provider failure when a sync caller publishes while an async caller waits.

Execution-path integration follows in later layers; no public API or dependency changes.

Issues

Relates to #3672. Split from #4585; neither issue nor original PR is closed by this layer.

Testing

Moved all source GetKeyAsync_* coverage into SymmetricKeyCacheTests, including sync/async identity, cancellation, provider serialization, failed-owner retry, and zero-TTL concurrency. Split behavior-switch theories and combined scenarios into separate facts; provider fixtures live under UnitTests/Utilities. Added a regression for synchronous publication while async waits for the provider gate.

Commands run on macOS after the self-review fix:

dotnet test src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj --framework net10.0 --configuration Debug --no-build --no-restore --list-tests --filter 'FullyQualifiedName~SymmetricKeyCacheTests' --verbosity quiet
# Passed: 17 tests discovered.

dotnet test src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj --framework net10.0 --configuration Debug --no-restore --filter 'FullyQualifiedName~SymmetricKeyCacheTests' --blame-hang-timeout 2m --verbosity quiet
# Passed: 17, skipped: 0.

dotnet test src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj --framework net10.0 --configuration Release --no-restore --filter 'FullyQualifiedName~SymmetricKeyCacheTests' --blame-hang-timeout 2m --verbosity quiet
# Passed: 17, skipped: 0.

dotnet build src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj --framework net462 --configuration Debug --no-restore --verbosity quiet
# Passed: 0 warnings, 0 errors.

dotnet build src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj --framework net462 --configuration Release --no-restore --verbosity quiet
# Passed: 0 warnings, 0 errors.

dotnet build build.proj -t:BuildSqlClient -p:Configuration=Debug -verbosity:quiet
# Passed: implementation, reference, unsupported-platform builds; 0 warnings/errors.

dotnet build build.proj -t:BuildSqlClient -p:Configuration=Release -verbosity:quiet
# Passed: implementation, reference, unsupported-platform builds; 0 warnings/errors.

The new race regression failed before the fix with the redundant provider-call exception, then passed in both suites above:

dotnet test src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj --framework net10.0 --configuration Debug --no-build --no-restore --filter 'FullyQualifiedName~GetKeyAsync_SynchronousOwnerPublishesWhileWaiting_ReusesPublishedKey' --blame-hang-timeout 2m --verbosity minimal
# Before fix: 1 failed, as expected.

No SQL Server integration tests or Windows/.NET Framework runtime tests were run. This layer has no execution-path callers yet; server coverage belongs with integration layers.

  • Tests added or updated
  • Public API changes documented (none)
  • Verified against customer repro (not applicable to this foundation layer)
  • Ensure no breaking changes introduced

Guidelines

Please review the contribution guidelines before submitting a pull request:

Extract the cache foundation from #4585 with per-key async loading, cancellation, and shared sync/async publication. Relocate cache regressions into focused cache tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 18:59
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Nov 16, 2026
@cheenamalhotra cheenamalhotra added this to the 8.1.0-preview1 milestone Nov 16, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human verification is needed because the current PR identity, feedback and CI evidence, and a trusted revision of supplemental review policies were not established.

No open issues 🎉

What changed in this PR

Adds the asynchronous Always Encrypted symmetric-key cache foundation for #3672, as the first of five layers split from #4585. Command execution integration is deferred; this layer changes no public APIs or dependencies.

Changes:

  • Adds per-key load gates, cancellable waits, and concurrent uncached loads when the cache lifetime is zero.
  • Shares sync/async cache publication while serializing provider invocation without holding the gate during outstanding async I/O.
  • Adds 16 unit tests covering cache reuse, cancellation, retries, provider serialization, and publication races.
File Description
src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​Utilities/​SymmetricKeyCacheTestProviders.cs Adds observable synchronous and asynchronous test providers.
src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​Utilities/​PendingDecryptionKeyStoreProvider.cs Adds a controllable provider for overlapping decryption tests.
src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​TestIsolationCollections.cs Isolates tests that change the process-wide cache lifetime.
src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​Microsoft/​Data/​SqlClient/​AlwaysEncrypted/​SymmetricKeyCacheTests.cs Adds async cache behavior and concurrency regression coverage.
src/​Microsoft.Data.SqlClient/​src/​Microsoft/​Data/​SqlClient/​AlwaysEncrypted/​SymmetricKeyCache.cs Implements async loading, per-key coordination, and shared cache publication.

Review effort: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 19:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Required policy checks could not be completed because trusted supplemental review policies or references were unavailable, and head CI is still pending.

No open issues 🎉

Review effort: Balanced

Avoid repeating decryption when a synchronous caller publishes the key while an async caller waits for the provider gate. Cover the race with a deterministic regression test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 19:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Required policy checks could not be completed without trusted review references, while head CI remains pending and Windows/.NET Framework runtime coverage is unconfirmed.

No open issues 🎉

Review effort: Balanced

@codecov

codecov Bot commented Nov 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.46018% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.07%. Comparing base (17bef7b) to head (16919af).
⚠️ Report is 10 commits behind head on main.

Files with missing lines Patch % Lines
...ata/SqlClient/AlwaysEncrypted/SymmetricKeyCache.cs 96.46% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5077      +/-   ##
==========================================
+ Coverage   68.99%   70.07%   +1.08%     
==========================================
  Files         314      314              
  Lines       45857    69893   +24036     
==========================================
+ Hits        31638    48981   +17343     
- Misses      14219    20912    +6693     
Flag Coverage Δ
CI-SqlClient ?
sqlclient-pr 70.07% <96.46%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: To triage

Development

Successfully merging this pull request may close these issues.

2 participants