fix: Preserve int64 precision when rebuilding nested values from strings - #2553
Merged
Conversation
erezrokah
marked this pull request as ready for review
August 3, 2026 15:03
This was referenced Aug 3, 2026
Merged
stoovon
approved these changes
Aug 3, 2026
kodiakhq Bot
pushed a commit
that referenced
this pull request
Aug 3, 2026
🤖 I have created a release *beep* *boop* --- ## [4.96.0](v4.95.3...v4.96.0) (2026-08-03) ### Features * Expose error classifier ([#2539](#2539)) ([e042d73](e042d73)) ### Bug Fixes * **deps:** Update aws-sdk-go-v2 monorepo ([#2521](#2521)) ([cc5ce53](cc5ce53)) * **deps:** Update aws-sdk-go-v2 monorepo ([#2524](#2524)) ([d080fbf](d080fbf)) * **deps:** Update aws-sdk-go-v2 monorepo ([#2537](#2537)) ([f5b5033](f5b5033)) * **deps:** Update aws-sdk-go-v2 monorepo ([#2540](#2540)) ([579a722](579a722)) * **deps:** Update aws-sdk-go-v2 monorepo ([#2542](#2542)) ([83ceb77](83ceb77)) * **deps:** Update aws-sdk-go-v2 monorepo ([#2548](#2548)) ([6be63a3](6be63a3)) * **deps:** Update github.com/cloudquery/jsonschema digest to 45e7e20 ([#2535](#2535)) ([acb4b1b](acb4b1b)) * **deps:** Update go module directive to v1.26.4 ([#2528](#2528)) ([66974c9](66974c9)) * **deps:** Update go module directive to v1.26.5 ([#2547](#2547)) ([cb54f5d](cb54f5d)) * **deps:** Update golang.org/x/exp digest to c48552f ([#2527](#2527)) ([4b94b83](4b94b83)) * **deps:** Update module github.com/apache/arrow-go/v18 to v18.7.0 ([#2544](#2544)) ([301cb8c](301cb8c)) * **deps:** Update module github.com/cloudquery/cloudquery-api-go to v1.14.12 ([#2516](#2516)) ([2107281](2107281)) * **deps:** Update module github.com/getsentry/sentry-go to v0.47.0 ([#2529](#2529)) ([94da8e3](94da8e3)) * **deps:** Update module github.com/getsentry/sentry-go to v0.48.0 ([#2549](#2549)) ([133be1f](133be1f)) * **deps:** Update module golang.org/x/text to v0.40.0 ([#2550](#2550)) ([dc3e10e](dc3e10e)) * **deps:** Update module google.golang.org/grpc to v1.82.0 ([#2538](#2538)) ([aee8e3f](aee8e3f)) * **deps:** Update module google.golang.org/grpc to v1.82.1 ([#2543](#2543)) ([7182a82](7182a82)) * **deps:** Update opentelemetry-go monorepo ([#2532](#2532)) ([90f446e](90f446e)) * Preserve int64 precision when rebuilding nested values from strings ([#2553](#2553)) ([8599fa0](8599fa0)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kodiakhq Bot
pushed a commit
that referenced
this pull request
Aug 4, 2026
…2556) Follow-up to #2553, which fixed the write-test helpers. The `scalar` package has the same problem on the read side: `Struct.Set` and `List.Set` decode JSON into `map[string]any`/`[]any` with the default decoder, so int64/uint64 values beyond float64 precision are rounded before they ever reach a builder. `{"v":-8717895732742165505}` came back as `{"v":-8717895732742166000}`, and `[-8717895732742165505]` as `[-8717895732742165504]`. Decode with `UseNumber` and teach `Int`, `Uint` and `Float` to accept `json.Number` (delegating to the existing string parse — `json.Number` is a distinct named type, so the existing `case string` never matched it). This is what still blocks the postgresql destination in cloudquery/cloudquery#23163, whose read path goes through `scalar.NewScalar`/`Set`.
erezrokah
added a commit
to cloudquery/cloudquery
that referenced
this pull request
Aug 4, 2026
…23163) This PR contains the following updates: | Package | Change | |---|---| | [github.com/apache/arrow-go/v18](https://github.com/apache/arrow-go) | `v18.6.0` → `v18.7.0` | | [github.com/cloudquery/plugin-sdk/v4](https://github.com/cloudquery/plugin-sdk) | `v4.95.3` → `v4.96.1` | | [github.com/cloudquery/filetypes/v4](https://github.com/cloudquery/filetypes) | `v4.7.1` → `v4.7.3` | arrow-go v18.7.0 made `array.FromJSON` decode int64/uint64 exactly. Every other JSON path that rebuilds nested values still went through float64, so round-trips silently rounded (`-8717895732742165505` became `-8717895732742165504`) and the mismatch broke the write test suite across all Go destinations. Fixed upstream: - plugin-sdk v4.96.0 (cloudquery/plugin-sdk#2553) — shared write-test helpers - plugin-sdk v4.96.1 (cloudquery/plugin-sdk#2556) — the `scalar` package, which postgresql reads through - filetypes v4.7.3 (cloudquery/filetypes#757) — file-based destinations Fixed here, by decoding nested values with `UseNumber`: - 16 `AppendValueFromString` call sites across 12 destinations - `UnmarshalOne` decoders in bigquery, elasticsearch, meilisearch, mongodb - postgresql's `stripNullsFromMarshalledJson` - mongodb's struct/JSON write path, converting `json.Number` to the integer types the BSON encoder accepts, and its read path, restoring nested uint64 values stored as int64 bits - elasticsearch and meilisearch document decodes, with numeric builders taught to accept `json.Number` - two float64-rounded literals in the transformer/basic test azblob fails with `AccountIsDisabled`, which also fails on main and is unrelated to this PR. Supersedes #23230, whose plugin-sdk bump is included here — it fails on its own because v4.96.0 requires arrow v18.7.0. --------- Co-authored-by: cloudquery-ci[bot] <271027272+cloudquery-ci[bot]@users.noreply.github.com> Co-authored-by: erezrokah <erezrokah@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
arrow-go v18.7.0 made
array.FromJSONdecode int64/uint64 exactly, butAppendValueFromStringstill routes nested values through float64, so list/struct round-trips silently lose precision (-8717895732742165505becomes-8717895732742165504) and every destination plugin's write test suite fails. Route nested types throughUnmarshalOnewithUseNumberinstead.