Skip to content

fix(json): only convert unambiguous date/datetime/UUID strings in additional data#664

Open
56badkarma wants to merge 1 commit into
microsoft:mainfrom
56badkarma:fix/json-additional-data-string-coercion
Open

fix(json): only convert unambiguous date/datetime/UUID strings in additional data#664
56badkarma wants to merge 1 commit into
microsoft:mainfrom
56badkarma:fix/json-additional-data-string-coercion

Conversation

@56badkarma

Copy link
Copy Markdown

Overview

JsonParseNode.try_get_anything runs every untyped string (everything landing in additional_data) through five lenient parsers in sequence: datetime_from_iso_format_compat, UUID, parse_timedelta_string, date.fromisoformat, time_from_iso_format_compat. On Python 3.11+ the lenient parsers accept many ordinary text values:

'19-2026'       -> datetime.time(19, 0, tzinfo=timezone(timedelta(days=-1, seconds=12840)))  # invoice number
'100-012863299' -> datetime.time(10, 0, ...)                                                 # invoice number
'PT'            -> datetime.timedelta(0)                                                     # country code
'10:30'         -> datetime.timedelta(seconds=37800)
'11.0'          -> datetime.time(11, 0)                                                      # version string
'd41d8cd98f00b204e9800998ecf8427e' -> UUID(...)                                              # 32-hex string

This is not just a display problem: in our production SharePoint AP-invoicing app, invoice numbers like 19-2026 were coerced to time objects, later stringified on update, and persisted back — permanently corrupting stored data (details with a captured wire payload in microsoftgraph/msgraph-sdk-python#1340).

This PR replaces the speculative parsing with two anchored patterns, mirroring the .NET JsonParseNode.TryGetAnything, which only attempts strict DateTime/DateTimeOffset/Guid parses and otherwise returns the string:

  • full ISO date or datetime (^\d{4}-\d{2}-\d{2}([Tt ]\d{2}:\d{2}|$)) → datetime_from_iso_format_compat
  • canonical hyphenated UUID → UUID

Behavior is unchanged for real payload values: full ISO datetimes still become datetime, date-only strings still become datetime (same as today, via the same compat parser), canonical UUIDs still become UUID. The timedelta/time guessing is removed entirely — Graph does not send bare durations or times as untyped strings, and their grammars overlap with too much real-world text. The __is_four_digit_number/isdigit() special cases (added for earlier symptom reports like microsoftgraph/msgraph-sdk-python#653) become unnecessary since all-digit strings cannot match either anchored pattern, and are removed.

Related Issue

Fixes #487
Addresses microsoftgraph/msgraph-sdk-python#1340 and microsoftgraph/msgraph-sdk-python#653

Notes

  • Strings that previously coerced to timedelta/time/unhyphenated-UUID will now stay strings in additional_data. That is the intent of the fix, but callers who relied on those accidental conversions would see a change; typed model fields are unaffected (they go through declared field deserializers and never reach try_get_anything).
  • Verified against a live Microsoft Graph SharePoint tenant: text columns round-trip unchanged, Created/Modified datetimes still parse.

Testing Instructions

  • cd packages/serialization/json && pytest tests/
  • New parametrized test test_get_anything_does_not_coerce_text_values covers each reported corruption pattern (19-2026, PT, 10:30, 23.085, 11.0, 32-hex, date fragments)
  • New tests pin the preserved conversions: date-only → datetime, canonical UUID → UUID; existing tests cover full ISO datetimes

🤖 Generated with Claude Code

https://claude.ai/code/session_019Uui4g7E7UmzKRGHMW5zbH

…itional data

try_get_anything ran every untyped string through five lenient parsers
(datetime, UUID, timedelta, date, time). On Python 3.11+ this mangled
ordinary text values landing in additional_data: "19-2026" became
time(19, 0) at offset -20:26, "PT" became timedelta(0), "11.0" became
time(11, 0), and 32-char hex strings became UUIDs. Round-tripping such
records then persisted the mangled strings, corrupting user data.

Only convert strings matching an anchored full ISO date/datetime shape
or the canonical hyphenated UUID form, mirroring the .NET JsonParseNode
which attempts strict DateTime/DateTimeOffset/Guid parses only. Date-only
strings still parse through datetime_from_iso_format_compat, preserving
the existing datetime result for real payload values. The previous
four-digit and isdigit special cases are no longer needed since all-digit
strings cannot match either pattern.

Fixes microsoft#487

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Uui4g7E7UmzKRGHMW5zbH
@sonarqubecloud

Copy link
Copy Markdown

@56badkarma

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

try_get_anything - converts a 32 hex string to UUID

1 participant