fix: Preserve int64 precision when setting nested scalars from JSON - #2556
Merged
Conversation
murarustefaan
approved these changes
Aug 4, 2026
kodiakhq Bot
pushed a commit
that referenced
this pull request
Aug 4, 2026
🤖 I have created a release *beep* *boop* --- ## [4.96.1](v4.96.0...v4.96.1) (2026-08-04) ### Bug Fixes * Preserve int64 precision when setting nested scalars from JSON ([#2556](#2556)) ([65ae9bc](65ae9bc)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
erezrokah
added a commit
to cloudquery/cloudquery
that referenced
this pull request
Aug 4, 2026
…23163) This PR contains the following updates: | Package | Change | |---|---| | [github.com/apache/arrow-go/v18](https://github.com/apache/arrow-go) | `v18.6.0` → `v18.7.0` | | [github.com/cloudquery/plugin-sdk/v4](https://github.com/cloudquery/plugin-sdk) | `v4.95.3` → `v4.96.1` | | [github.com/cloudquery/filetypes/v4](https://github.com/cloudquery/filetypes) | `v4.7.1` → `v4.7.3` | arrow-go v18.7.0 made `array.FromJSON` decode int64/uint64 exactly. Every other JSON path that rebuilds nested values still went through float64, so round-trips silently rounded (`-8717895732742165505` became `-8717895732742165504`) and the mismatch broke the write test suite across all Go destinations. Fixed upstream: - plugin-sdk v4.96.0 (cloudquery/plugin-sdk#2553) — shared write-test helpers - plugin-sdk v4.96.1 (cloudquery/plugin-sdk#2556) — the `scalar` package, which postgresql reads through - filetypes v4.7.3 (cloudquery/filetypes#757) — file-based destinations Fixed here, by decoding nested values with `UseNumber`: - 16 `AppendValueFromString` call sites across 12 destinations - `UnmarshalOne` decoders in bigquery, elasticsearch, meilisearch, mongodb - postgresql's `stripNullsFromMarshalledJson` - mongodb's struct/JSON write path, converting `json.Number` to the integer types the BSON encoder accepts, and its read path, restoring nested uint64 values stored as int64 bits - elasticsearch and meilisearch document decodes, with numeric builders taught to accept `json.Number` - two float64-rounded literals in the transformer/basic test azblob fails with `AccountIsDisabled`, which also fails on main and is unrelated to this PR. Supersedes #23230, whose plugin-sdk bump is included here — it fails on its own because v4.96.0 requires arrow v18.7.0. --------- Co-authored-by: cloudquery-ci[bot] <271027272+cloudquery-ci[bot]@users.noreply.github.com> Co-authored-by: erezrokah <erezrokah@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #2553, which fixed the write-test helpers. The
scalarpackage has the same problem on the read side:Struct.SetandList.Setdecode JSON intomap[string]any/[]anywith the default decoder, so int64/uint64 values beyond float64 precision are rounded before they ever reach a builder.{"v":-8717895732742165505}came back as{"v":-8717895732742166000}, and[-8717895732742165505]as[-8717895732742165504].Decode with
UseNumberand teachInt,UintandFloatto acceptjson.Number(delegating to the existing string parse —json.Numberis a distinct named type, so the existingcase stringnever matched it).This is what still blocks the postgresql destination in cloudquery/cloudquery#23163, whose read path goes through
scalar.NewScalar/Set.