Skip to content

Commit e768e81

Browse files
committed
Merge remote-tracking branch 'upstream/main' into fix/check-connection-no-wait
# Conflicts: # src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/ManagedSni/SniTcpHandleTests.cs
2 parents 51098c5 + 44be992 commit e768e81

52 files changed

Lines changed: 1095 additions & 222 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.config/LocalizationValidationAllowlist.json renamed to .config/LocalizationValidationAllowlist.jsonc

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,19 @@
11
{
2-
"_comment": [
3-
"These culture/key pairs intentionally match the English source text.",
4-
"Each pair was verified in the internal LCL source as localized (Stat=Loc, Orig=New). Remove an entry when its localized value changes."
5-
],
6-
"AllowedEnglishValueMatches": {
2+
// Schema: { "<resource owner>": { "<localized filename>": ["<resource key>", ...] } }
3+
// Owners: SqlClient (driver resources) and Abstractions (authentication resources).
4+
// Each owner maps Strings.<culture>.resx filenames to arrays of approved resource keys.
5+
// An empty owner object means no exceptions. Omit files that need no exceptions.
6+
// The validator reads only the selected owner; a missing owner section is an error.
7+
// Filenames and keys are case-sensitive and must exist in the selected resource set.
8+
// Keys must be non-empty, unique within each file's array, and have non-empty English values.
9+
// An exception permits only an exact English-value match in that one owner and culture.
10+
// It cannot exempt missing keys, empty translations, or keys absent from Strings.resx.
11+
//
12+
// These culture/key pairs intentionally match the English source text.
13+
// Each pair was verified in the internal LCL source as localized (Stat=Loc, Orig=New).
14+
// Remove an entry when its localized value changes.
15+
"Abstractions": {},
16+
"SqlClient": {
717
"Strings.cs.resx": [
818
"ADP_InvalidMultipartName",
919
"DataCategory_Data",

‎.github/instructions/onebranch-pipeline-design.instructions.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Respect this graph when modifying build stages:
3030

3131
Validation runs in three places and shares one gating switch.
3232

33-
- **Localization** — `steps/validate-localization-step.yml`, in the SqlClient build job before the driver is built. Reports missing or obsolete keys, empty localized values whose English value is non-empty, and untranslated resources. Approved identical translations are listed by culture and resource key in `.config/LocalizationValidationAllowlist.json`.
33+
- **Localization** — `steps/validate-localization-step.yml`, in the SqlClient and Abstractions build jobs before compilation. Callers explicitly supply the resource directory and allowlist section. Reports missing or obsolete keys, empty localized values whose English value is non-empty, and untranslated resources. Approved identical translations are nested by resource owner, culture, and resource key in `.config/LocalizationValidationAllowlist.jsonc`: `SqlClient` for the driver and `Abstractions` for Abstractions. Use JSON comments to document exceptions.
3434
- **XML documentation** — `eng/pipelines/common/steps/validate-xml-docs-step.yml`, shared with PR validation, in three modes: snippet sources before each snippet-consuming project is built (SqlClient, SqlServer, Abstractions, and Azure), generated documentation after documented projects are built (including the separate SqlClient reference-assembly output), and assembled packages in `package_validation`. Reports malformed documentation IDs, unresolved cross-references, and `lib/` vs `ref/` documentation-trimming defects.
3535
- **Packages** — `steps/validate-packages-step.yml`, in `package_validation`. Runs `tools/PackageValidator` across the whole drop so its cross-package version and dependency rules apply.
3636

‎BUILDGUIDE.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -495,6 +495,53 @@ request submission, not symbol availability. With the default setting, the
495495
existing status check rejects terminal failures but does not wait for pending
496496
requests to complete.
497497

498+
## Localization Validation
499+
500+
OneBranch validates localized resources before building the SqlClient and Abstractions
501+
assemblies. Each caller explicitly supplies its resource directory and isolated allowlist
502+
section in [LocalizationValidationAllowlist.jsonc](.config/LocalizationValidationAllowlist.jsonc):
503+
`SqlClient` for the driver and `Abstractions` for Abstractions. Each owner contains
504+
localized filenames mapped to lists of resource keys approved to match English.
505+
The script defaults to `SqlClient` for direct callers.
506+
The JSONC file supports comments for documenting approved exceptions.
507+
508+
### Allowlist schema
509+
510+
The file is a JSON object with three levels:
511+
512+
1. **Resource owner**: `SqlClient` or `Abstractions`, selecting an independent exception set.
513+
2. **Localized resource filename**: `Strings.<culture>.resx`, relative to that owner's
514+
resource directory. Filenames are case-sensitive.
515+
3. **Resource keys**: an array of case-sensitive key names approved to have localized
516+
values exactly equal to their non-empty English values in `Strings.resx`.
517+
518+
For example, this abbreviated configuration permits `DataCategory_Xml` to remain
519+
identical to English in the driver's French resources, but grants no Abstractions exceptions:
520+
521+
```jsonc
522+
{
523+
"SqlClient": {
524+
// Approved for this resource owner and culture only.
525+
"Strings.fr.resx": ["DataCategory_Xml"]
526+
},
527+
"Abstractions": {}
528+
}
529+
```
530+
531+
Use an empty object (`{}`) for an owner with no exceptions, and omit filenames that need
532+
none. The selected owner section must exist; the validator never borrows another owner's
533+
exceptions. Each filename must exist in the selected resource directory. Each listed key
534+
must be non-empty, unique within its array, and present with a non-empty English value.
535+
536+
Exceptions permit English-value matches only: they do not excuse missing resource keys,
537+
empty translations, or keys absent from the English resource file. Remove an exception
538+
when its localized value changes; stale exceptions are reported as validation findings.
539+
Only add verified, intentional English matches, not strings still awaiting translation.
540+
541+
The existing `failOnValidationError` flag controls whether findings fail the build or
542+
are reported as warnings. Missing or malformed validator inputs always fail. Do not
543+
allowlist untranslated development strings merely to make validation pass.
544+
498545
## Versioning
499546

500547
Versioning can be accomplished by using a mix of different parameters to the `build.proj` targets:

‎eng/pipelines/onebranch/jobs/build-buildproj-job.yml‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -153,11 +153,19 @@ jobs:
153153
steps:
154154
- template: /eng/pipelines/onebranch/steps/script-output-environment-variables-step.yml@self
155155

156-
# Localized resources ship with the SqlClient driver. Validate them before analysis and
157-
# building so missing or untranslated strings fail every SqlClient build.
156+
# Validate each resource-owning assembly before analysis and building.
158157
- ${{ if eq(parameters.packageShortName, 'SqlClient') }}:
159158
- template: /eng/pipelines/onebranch/steps/validate-localization-step.yml@self
160159
parameters:
160+
resourcesDirectory: src/Microsoft.Data.SqlClient/src/Resources
161+
allowlistSection: SqlClient
162+
failOnValidationError: ${{ parameters.failOnValidationError }}
163+
164+
- ${{ if eq(parameters.packageShortName, 'Abstractions') }}:
165+
- template: /eng/pipelines/onebranch/steps/validate-localization-step.yml@self
166+
parameters:
167+
resourcesDirectory: src/Microsoft.Data.SqlClient.Extensions/Abstractions/src/Resources
168+
allowlistSection: Abstractions
161169
failOnValidationError: ${{ parameters.failOnValidationError }}
162170

163171
# Validate the documentation snippet sources. This requires no build, so a malformed

‎eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,24 @@ BeforeAll {
4949
}
5050

5151
Describe 'Validation step templates' {
52+
It 'selects the resource owner allowlist for <Owner>' -ForEach @(
53+
@{ Owner = 'SqlClient'; Directory = 'src/Microsoft.Data.SqlClient/src/Resources' }
54+
@{ Owner = 'Abstractions'; Directory = 'src/Microsoft.Data.SqlClient.Extensions/Abstractions/src/Resources' }
55+
) {
56+
$content = Get-Content -LiteralPath (Join-Path $script:jobsPath 'build-buildproj-job.yml') -Raw
57+
$allowlistPath = Join-Path $script:repoRoot '.config/LocalizationValidationAllowlist.jsonc'
58+
$allowlist = Get-Content -LiteralPath $allowlistPath -Raw | ConvertFrom-Json
59+
$step = Get-Content -LiteralPath (
60+
Join-Path $script:stepsPath 'validate-localization-step.yml') -Raw
61+
62+
$step | Should -Match '\.config/LocalizationValidationAllowlist\.jsonc"'
63+
$content | Should -Match (
64+
'resourcesDirectory: ' + [regex]::Escape($Directory) +
65+
'\r?\n\s+allowlistSection: ' + [regex]::Escape($Owner) + '\r?\n')
66+
$allowlist.PSObject.Properties.Name | Should -Contain $Owner
67+
Test-Path -LiteralPath (Join-Path $script:repoRoot $Directory) | Should -BeTrue
68+
}
69+
5270
It 'never passes a switch using the -Switch:Value form' -ForEach @(
5371
@{ Template = 'validate-localization-step.yml' }
5472
@{ Template = 'validate-packages-step.yml' }

‎eng/pipelines/onebranch/scripts/tests/validate-localization.Tests.ps1‎

Lines changed: 70 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,70 @@ BeforeAll {
3636
}
3737

3838
Describe 'validate-localization.ps1' {
39+
It 'accepts JSON comments in the <Owner> allowlist' -ForEach @(
40+
@{ Owner = 'SqlClient' }
41+
@{ Owner = 'Abstractions' }
42+
) {
43+
$resources = New-ResourcesDirectory
44+
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' }
45+
Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Greeting = 'Hello' }
46+
$allowlist = Join-Path $resources 'allowlist.jsonc'
47+
@'
48+
{
49+
// Approved English matches, isolated by resource owner.
50+
"SqlClient": { "Strings.fr.resx": ["Greeting"] },
51+
/* Abstractions has an independent set of approved matches. */
52+
"Abstractions": { "Strings.fr.resx": ["Greeting"] }
53+
}
54+
'@ | Set-Content -LiteralPath $allowlist
55+
56+
{ & $scriptPath -ResourcesDirectory $resources -AllowlistPath $allowlist `
57+
-AllowlistSection $Owner } | Should -Not -Throw
58+
}
59+
60+
It 'selects an independent allowlist for a second resource assembly' {
61+
$resources = New-ResourcesDirectory
62+
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' }
63+
Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Greeting = 'Bonjour' }
64+
$allowlist = Join-Path $resources 'allowlist.json'
65+
@{
66+
SqlClient = @{ 'Strings.fr.resx' = @('UnrelatedDriverKey') }
67+
Abstractions = @{}
68+
} | ConvertTo-Json -Depth 4 | Set-Content -LiteralPath $allowlist
69+
{ & $scriptPath -ResourcesDirectory $resources -AllowlistPath $allowlist `
70+
-AllowlistSection Abstractions } | Should -Not -Throw
71+
{ & $scriptPath -ResourcesDirectory $resources -AllowlistPath $allowlist `
72+
-AllowlistSection SqlClient } |
73+
Should -Throw '*Localization validation failed with 1 error*'
74+
}
75+
76+
It 'rejects a missing resource-set allowlist section rather than borrowing another assembly allowlist' {
77+
$resources = New-ResourcesDirectory
78+
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' }
79+
Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Greeting = 'Bonjour' }
80+
$allowlist = Join-Path $resources 'allowlist.json'
81+
@{ SqlClient = @{} } | ConvertTo-Json | Set-Content -LiteralPath $allowlist
82+
{ & $scriptPath -ResourcesDirectory $resources -AllowlistPath $allowlist `
83+
-AllowlistSection Abstractions } |
84+
Should -Throw "*must define 'Abstractions'*"
85+
}
86+
87+
It 'honors an English-match exception only in the selected resource-set section' {
88+
$resources = New-ResourcesDirectory
89+
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' }
90+
Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Greeting = 'Hello' }
91+
$allowlist = Join-Path $resources 'allowlist.json'
92+
@{
93+
SqlClient = @{}
94+
Abstractions = @{ 'Strings.fr.resx' = @('Greeting') }
95+
} | ConvertTo-Json -Depth 4 | Set-Content -LiteralPath $allowlist
96+
97+
{ & $scriptPath -ResourcesDirectory $resources -AllowlistPath $allowlist `
98+
-AllowlistSection Abstractions } | Should -Not -Throw
99+
{ & $scriptPath -ResourcesDirectory $resources -AllowlistPath $allowlist } |
100+
Should -Throw '*Localization validation failed with 1 error*'
101+
}
102+
39103
It 'accepts complete localized files with translated values' {
40104
$resources = New-ResourcesDirectory
41105
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello'; Farewell = 'Goodbye' }
@@ -105,7 +169,7 @@ Describe 'validate-localization.ps1' {
105169
$allowlist = Join-Path $resources 'allowlist.json'
106170
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' }
107171
Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Greeting = 'Hello' }
108-
@{ AllowedEnglishValueMatches = @{ 'Strings.fr.resx' = @('Greeting') } } |
172+
@{ SqlClient = @{ 'Strings.fr.resx' = @('Greeting') } } |
109173
ConvertTo-Json -Depth 5 |
110174
Set-Content -LiteralPath $allowlist
111175

@@ -118,7 +182,7 @@ Describe 'validate-localization.ps1' {
118182
$allowlist = Join-Path $resources 'allowlist.json'
119183
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello'; Farewell = 'Goodbye' }
120184
Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Greeting = 'Bonjour' }
121-
@{ AllowedEnglishValueMatches = @{ 'Strings.fr.resx' = @('Farewell') } } |
185+
@{ SqlClient = @{ 'Strings.fr.resx' = @('Farewell') } } |
122186
ConvertTo-Json -Depth 5 |
123187
Set-Content -LiteralPath $allowlist
124188

@@ -131,7 +195,7 @@ Describe 'validate-localization.ps1' {
131195
$allowlist = Join-Path $resources 'allowlist.json'
132196
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' }
133197
Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Greeting = 'Bonjour' }
134-
@{ AllowedEnglishValueMatches = @{ 'Strings.fr.resx' = @('Unknown') } } |
198+
@{ SqlClient = @{ 'Strings.fr.resx' = @('Unknown') } } |
135199
ConvertTo-Json -Depth 5 |
136200
Set-Content -LiteralPath $allowlist
137201

@@ -145,7 +209,7 @@ Describe 'validate-localization.ps1' {
145209
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' }
146210
Set-ResourceFile (Join-Path $resources 'Strings.de.resx') @{ Greeting = 'Hello' }
147211
Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Greeting = 'Hello' }
148-
@{ AllowedEnglishValueMatches = @{ 'Strings.de.resx' = @('Greeting') } } |
212+
@{ SqlClient = @{ 'Strings.de.resx' = @('Greeting') } } |
149213
ConvertTo-Json -Depth 5 |
150214
Set-Content -LiteralPath $allowlist
151215

@@ -167,7 +231,7 @@ Describe 'validate-localization.ps1' {
167231
$allowlist = Join-Path $resources 'allowlist.json'
168232
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' }
169233
Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Greeting = 'Bonjour' }
170-
@{ AllowedEnglishValueMatches = @{ 'Strings.fr.resx' = @('Greeting') } } |
234+
@{ SqlClient = @{ 'Strings.fr.resx' = @('Greeting') } } |
171235
ConvertTo-Json -Depth 5 |
172236
Set-Content -LiteralPath $allowlist
173237

@@ -180,7 +244,7 @@ Describe 'validate-localization.ps1' {
180244
$allowlist = Join-Path $resources 'allowlist.json'
181245
Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Unused = '' }
182246
Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Unused = '' }
183-
@{ AllowedEnglishValueMatches = @{ 'Strings.fr.resx' = @('Unused') } } |
247+
@{ SqlClient = @{ 'Strings.fr.resx' = @('Unused') } } |
184248
ConvertTo-Json -Depth 5 |
185249
Set-Content -LiteralPath $allowlist
186250

‎eng/pipelines/onebranch/scripts/validate-localization.ps1‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,16 @@
66
Directory containing the English and localized Strings.resx files.
77
88
.PARAMETER AllowlistPath
9-
Optional JSON file containing approved English-value matches grouped by localized filename.
9+
Optional JSONC file containing approved English-value matches grouped by resource owner,
10+
localized filename, and resource key.
1011
1112
.PARAMETER ReportOnly
1213
Report validation findings as warnings without failing. Malformed or missing inputs still fail,
1314
because a run that could not examine the resources has produced no result to report.
15+
16+
.PARAMETER AllowlistSection
17+
Resource owner containing the allowlist for the resource set being validated.
18+
Defaults to SqlClient.
1419
#>
1520

1621
# Licensed to the .NET Foundation under one or more agreements.
@@ -25,7 +30,9 @@ param(
2530

2631
[string]$AllowlistPath,
2732

28-
[switch]$ReportOnly
33+
[switch]$ReportOnly,
34+
35+
[string]$AllowlistSection = 'SqlClient'
2936
)
3037

3138
Set-StrictMode -Version Latest
@@ -83,9 +90,9 @@ if (-not [string]::IsNullOrWhiteSpace($AllowlistPath)) {
8390
}
8491

8592
$configuration = Get-Content -LiteralPath $AllowlistPath -Raw | ConvertFrom-Json
86-
$englishValueMatchesProperty = $configuration.PSObject.Properties['AllowedEnglishValueMatches']
93+
$englishValueMatchesProperty = $configuration.PSObject.Properties[$AllowlistSection]
8794
if ($null -eq $englishValueMatchesProperty) {
88-
throw "Localization allowlist file '$AllowlistPath' must define 'AllowedEnglishValueMatches'."
95+
throw "Localization allowlist file '$AllowlistPath' must define '$AllowlistSection'."
8996
}
9097

9198
foreach ($fileProperty in $englishValueMatchesProperty.Value.PSObject.Properties) {

‎eng/pipelines/onebranch/steps/validate-localization-step.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,11 @@ parameters:
99
# the step succeeds.
1010
- name: failOnValidationError
1111
type: boolean
12+
# Each resource owner supplies its directory and isolated exception set explicitly.
13+
- name: resourcesDirectory
14+
type: string
15+
- name: allowlistSection
16+
type: string
1217

1318
steps:
1419
# Compose the argument line first so the optional switch can be appended without repeating the
@@ -21,8 +26,9 @@ steps:
2126
# other than True or False is rejected rather than assumed, so an unrecognised value cannot
2227
# silently downgrade validation to warnings.
2328
- pwsh: |
24-
$arguments = '-ResourcesDirectory "$(Build.SourcesDirectory)/src/Microsoft.Data.SqlClient/src/Resources"' +
25-
' -AllowlistPath "$(Build.SourcesDirectory)/.config/LocalizationValidationAllowlist.json"'
29+
$arguments = '-ResourcesDirectory "$(Build.SourcesDirectory)/${{ parameters.resourcesDirectory }}"' +
30+
' -AllowlistPath "$(Build.SourcesDirectory)/.config/LocalizationValidationAllowlist.jsonc"' +
31+
' -AllowlistSection "${{ parameters.allowlistSection }}"'
2632
2733
$failOnValidationError = '${{ parameters.failOnValidationError }}'
2834
if ($failOnValidationError -eq 'False') {

‎eng/pipelines/onebranch/steps/validate-packages-step.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,9 @@ parameters:
5959
type: boolean
6060

6161
steps:
62+
- task: NuGetAuthenticate@1
63+
displayName: 'Authenticate NuGet feeds'
64+
6265
- task: DotNetCoreCLI@2
6366
displayName: 'build.proj - BuildPackageValidator'
6467
inputs:

‎src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ColumnEncryptionKeyInfo.cs‎

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,6 @@ namespace Microsoft.Data.SqlClient
1212
/// </summary>
1313
internal class ColumnEncryptionKeyInfo
1414
{
15-
internal readonly int KeyId;
16-
internal readonly int DatabaseId;
1715
internal readonly byte[] DecryptedKeyBytes;
1816
internal readonly byte[] KeyIdBytes;
1917
internal readonly byte[] DatabaseIdBytes;
@@ -34,7 +32,6 @@ internal class ColumnEncryptionKeyInfo
3432
/// <param name="keyid">key id for this column encryption key</param>
3533
internal ColumnEncryptionKeyInfo(byte[] decryptedKey, int databaseId, ulong keyMetadataVersion, int keyid)
3634
{
37-
3835
if (decryptedKey == null)
3936
{
4037
throw SQL.NullArgumentInConstructorInternal(_decryptedKeyName, _className);
@@ -44,8 +41,6 @@ internal ColumnEncryptionKeyInfo(byte[] decryptedKey, int databaseId, ulong keyM
4441
throw SQL.EmptyArgumentInConstructorInternal(_decryptedKeyName, _className);
4542
}
4643

47-
KeyId = keyid;
48-
DatabaseId = databaseId;
4944
DecryptedKeyBytes = decryptedKey;
5045
KeyMetadataVersion = keyMetadataVersion;
5146

@@ -101,7 +96,6 @@ internal int GetLengthForSerialization()
10196
/// <returns>next available offset</returns>
10297
internal int SerializeToBuffer(byte[] bytePackage, int startOffset)
10398
{
104-
10599
if (null == bytePackage)
106100
{
107101
throw SQL.NullArgumentInternal(_bytePackageName, _className, _serializeToBufferMethodName);

0 commit comments

Comments
 (0)