Skip to content

improved BigDecimal parsing performance - #3992

Open
wenshao wants to merge 1 commit into
mainfrom
dec_optim_20260216
Open

wenshao wants to merge 1 commit into
mainfrom
dec_optim_20260216

Conversation

@wenshao

@wenshao wenshao commented Feb 16, 2026

Copy link
Copy Markdown
Member
  • Use bit-shift operations for faster digit parsing
  • Simplify overflow detection to single check after digit loop
  • Handle exponent with combined scale to avoid string-based parsing

What this PR does / why we need it?

Summary of your change

Please indicate you've done the following:

  • Made sure tests are passing and test coverage is added if needed.
  • Made sure commit message follow the rule of Conventional Commits specification.
  • Considered the docs impact and opened a new docs issue or PR with docs changes if needed.

- Use bit-shift operations for faster digit parsing
- Simplify overflow detection to single check after digit loop
- Handle exponent with combined scale to avoid string-based parsing

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

@wenshao wenshao left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same overflow check is duplicated in core/src/main/java/com/alibaba/fastjson2/JSONReaderUTF16.java and should be fixed there as well.

— gpt-5.5 via Qwen Code /review

}
}

// Check overflow: more than 18 digits, or 19 digits with negative value (sign bit set)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] This post-loop overflow check can accept corrupted 19-digit values. For exactly 19 digits, values greater than Long.MAX_VALUE may overflow during longValue = longValue * 10 + digit and still finish with a non-negative wrapped longValue, so they take the BigDecimal.valueOf(longValue, combinedScale) fast path instead of falling back to TypeUtils.parseBigDecimal(...). That can silently return the wrong BigDecimal for valid JSON numbers outside signed long range.

A safer fix is to restore per-digit overflow detection, or add an exact 19-digit range check before using the accumulated longValue; otherwise force the existing string-based fallback.

— gpt-5.5 via Qwen Code /review

@wenshao wenshao left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed. Suggestions are inline.

— qwen3.8-max-preview via Qwen Code /review

Comment on lines +4416 to +4417
// Check overflow: more than 18 digits, or 19 digits with negative value (sign bit set)
boolean overflow = digitCount > 18 && (digitCount > 19 || longValue < 0);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The comment misdescribes the overflow condition — it reads as "more than 18 digits" is sufficient for overflow, but the code only flags overflow unconditionally at 20+ digits; 19 digits overflows only when the sign bit is set. — Concrete cost: a maintainer trusting the comment could simplify to digitCount > 18, forcing every valid 19-digit long (e.g. Long.MAX_VALUE) onto the slow TypeUtils.parseBigDecimal path — a silent performance regression.

Suggested change
// Check overflow: more than 18 digits, or 19 digits with negative value (sign bit set)
boolean overflow = digitCount > 18 && (digitCount > 19 || longValue < 0);
// Check overflow: 20+ digits always overflow; exactly 19 digits overflow only if sign bit is set
boolean overflow = digitCount > 18 && (digitCount > 19 || longValue < 0);

— qwen3.8-max-preview via Qwen Code /review

Comment on lines +6859 to +6860
// Check overflow: more than 18 digits, or 19 digits with negative value (sign bit set)
boolean overflow = digitCount > 18 && (digitCount > 19 || longValue < 0);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] Same misleading comment as in JSONReaderUTF16.java — it reads as "more than 18 digits" is sufficient for overflow, but the code only flags overflow unconditionally at 20+ digits; 19 digits overflows only when the sign bit is set. — Concrete cost: a maintainer trusting the comment could simplify to digitCount > 18, forcing every valid 19-digit long onto the slow path.

Suggested change
// Check overflow: more than 18 digits, or 19 digits with negative value (sign bit set)
boolean overflow = digitCount > 18 && (digitCount > 19 || longValue < 0);
// Check overflow: 20+ digits always overflow; exactly 19 digits overflow only if sign bit is set
boolean overflow = digitCount > 18 && (digitCount > 19 || longValue < 0);

— qwen3.8-max-preview via Qwen Code /review

Comment on lines +6953 to 6955
int combinedScale = scale - expValue;
decimal = BigDecimal.valueOf(negative ? -longValue : longValue, combinedScale);
value = true;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The new exponent fast path in JSONReaderUTF8.readBigDecimal() has no test asserting the exact BigDecimal result (value and scale) from byte[] input with exponent notation. The only UTF8 test with exponents (DoubleTest.parseDouble1, input "123.123E256") asserts only .doubleValue(), discarding scale. — Concrete cost: a future scale error in the UTF8 path that preserves the double value would go undetected. The UTF16 path is covered by Issue492 with exact BigDecimal assertions.

Consider adding a test that parses exponent-notation numbers from byte[] and asserts exact BigDecimal equality (e.g. assertEquals(new BigDecimal("1.1111111E7"), JSON.parseObject(bytes, BigDecimal.class))), mirroring Issue492's coverage.

— qwen3.8-max-preview via Qwen Code /review

qwen-code-dev-bot pushed a commit to wenshao/qwen-code that referenced this pull request Aug 3, 2026
* feat(cli): add a Java/JVM performance path rule to /review

The review's dimensions are domain-blind, and a Java diff's most
expensive regressions are decided by the JVM, not by anything visible
in the source: HotSpot chooses what to inline and what to compile by
the callee's bytecode size (MaxTrivialSize 6 / MaxInlineSize 35 /
FreqInlineSize 325 / HugeMethodLimit 8000), and a one-line change can
flip it on a hot path.

Like the GitHub Actions rule before it, the checklist attaches to
*.java paths and reaches every code-reviewing agent whose territory
contains one, scoped so non-Java diffs pay nothing. It carries:

- the correctness traps dressed as perf/concurrency code (shared
  SimpleDateFormat, two-call ConcurrentHashMap compounds, DCL without
  volatile) at Critical;
- the JVM-cost defects provable from source (per-call regex compiles,
  loop string +=, hot-path boxing, capturing lambdas in loops,
  unconditional log-message building, unpresized collections, legacy
  synchronized types, exceptions as control flow, per-call
  reflection) at Suggestion;
- the JIT inlining thresholds with a two-tier verification discipline:
  measure with javap against base and head (never estimate bytecode
  from source), or run -XX:+PrintInlining / JMH when the code is
  runnable; unmeasured inlining claims are reported as mechanism at
  low confidence.

Dogfooded against alibaba/fastjson2#3992 (BigDecimal parsing perf):
the performance agent applied the threshold reasoning correctly —
readBigDecimal was already far above FreqInlineSize before the diff
and the change shrinks it, so no crossing was possible and no
measurement was owed, stated with exactly that justification.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* feat(cli): name hot/cold splitting as the fix for a grown hot method

An A/B experiment measured what the checklist adds on top of the
model. Four crafted Java diffs (a hot method grown past FreqInlineSize,
a map-backed cache, a per-element append loop, a precompiled regex
parser) were each reviewed by a blind Agent-4 equivalent with and
without the Java path rule, on qwen3.8-max-preview.

Without the rule, the performance agent never once considered
inlining across all four diffs, and filed a high-confidence finding
that a constant long division costs 20-90 cycles per iteration —
bytecode-true, but C2 strength-reduces constant division to a
multiply-by-magic-number, so the cost does not survive the JIT. With
the rule, it measured every diff with javap (base 80 bytes, head 338,
crossing FreqInlineSize at 325), reported the crossing at low
confidence with the tier-2 check named, dismissed the division with
the correct mechanism, and proposed the fix as a hot/cold split with
the exact bytecode range to extract.

The fix shape is the part the model does not supply on its own, so
the checklist now names it: move cold paths into a private helper,
never @ForceInline (which bloats every caller), and state the
extraction as a bytecode range and a resulting size. The same
experiment cut three candidate additions — dense-key cache container
choice, per-element-to-bulk loops, and regex-for-fixed-formats —
because the control runs reached the same findings without them.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(cli): address review on the Java/JVM path rule

Seven findings from the PR review, all verified before fixing:

- The static measurement tier prescribed 'compile the base revision the
  same way', which reads as git checkout/stash in the one worktree nine
  agents share concurrently — or in the user's own checkout in local
  mode, where reviewsCode agents get no 'do not build the main checkout'
  guard (that guard is role-7-only). Rewritten as a non-mutating
  procedure: extract the base side with git show into a scratch dir,
  javac -d there, never checkout/stash/build in place.
- A full mvn/gradle build runs the branch's contributor-controlled build
  logic; the checklist now says to prefer javac on the extracted file
  and treat any build it does run as untrusted code (Agent 7's brief
  already carried this caveat; the rule extended the capability to nine
  agents without it).
- InlineSmallCode cited as ~1000 (the pre-JDK-11 value); measured 2500
  on a live JVM. HugeMethodLimit is a develop flag gated by the product
  DontCompileHugeMethods, and the boundary is > 8000, not >= 8000.
- 'Megamorphic -> no inlining at any size' overstated C2: a dominant
  receiver (TypeProfileMajorReceiverPercent, 90%) is still inlined
  behind a guard with an uncommon trap.
- pathRulesFor listed every triggering path in the heading of every
  agent's brief; a 200-file Java PR put ~11 KB of paths there. Capped at
  ten plus a count, for both rules.
- The flat 'performance findings are Suggestions' carried no escape
  hatch; added the one the workflow rule already needed — unbounded cost
  on attacker-reachable input is a DoS hole, graded Critical.
- Nits: the split fast-path excludes regex metacharacters (split(".")
  does not take it); test and generated sources are out of scope for the
  hot-path items; the Java match rows fold into the shared governed-table
  test so both rules assert through PATH_RULES.matches.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(cli): second-round review on the Java/JVM path rule

Six findings plus nits and three inline coverage probes, all verified:

- The 'safe' javac tier still executed contributor code: annotation
  processors on the classpath run at compile time. Add -proc:none
  (not optional), and make mvn/gradle a prohibition rather than a
  discouraged preference — on a stranger's PR branch the build logic is
  the attack surface.
- /tmp/<scratch> was a placeholder agents converge on; two compiling
  different revisions of one class into the same dir measure each
  other's bytecode. Prescribe SCRATCH=$(mktemp -d), with a %TEMP% note
  for Windows.
- 'Extract and javac' fails on any class with imports. Name the
  classpath path (-sourcepath at the module root, an existing
  target/classes, or mvn dependency:build-classpath which resolves
  without building) and a graceful fall-back to the mechanism tier
  instead of escalating to a project build.
- @ForceInline was ruled out for bloating callers — true but secondary,
  and the annotation is JDK-internal, not general. Lead with the
  anti-pattern that actually bites app code (reaching for -XX:FreqInlineSize
  / -XX:CompileCommand=inline, runtime knobs a PR cannot ship) and note
  @ForceInline only as unavailable.
- describePaths capped in diff order, so a test-heavy PR could name ten
  test files the rule scopes out and no production path. Stable-partition
  production first.
- Header: a rule earns its place by naming an invisible defect AND pays
  a per-agent token cost; say so, before rule #3 arrives.
- Nits: HashMap.newHashMap(n) (JDK 19+) for the presize arithmetic; the
  split fast path also covers the escaped two-char form.
- Coverage probes (inline): the correctness-traps block and the nine
  Suggestion patterns had zero test coverage — deletion left all tests
  green. Pin the load-bearing strings of both, plus the new tier flags
  and the production-first ordering.

* fix(cli): third-round review on the Java/JVM path rule

Drop the mvn dependency:build-classpath recommendation — Maven extensions
execute during any invocation, a strictly larger execution surface than
the annotation processors -proc:none exists to close. Fall through to the
mechanism tier when no pre-built classpath exists.

Add --release <N> at the project's target level: the same source compiles
to different bytecode at different levels (61 vs 16 bytes for a five-+
concatenation), so measuring without it produces a threshold verdict on
bytecode the shipped artifact does not contain.

Name -proc:none as a fidelity hazard: on a Lombok/Dagger project the
compiled class is missing generated members, so the static tier is void.

Add clauses for new files (no base side to compare), the base-side
target/classes caveat, and the Windows uniqueness primitive.

Fix the vacuous ordering assertion (indexOf returns -1 outside the cap,
and -1 < n passes). Add a GHA cap test and a --release/new-file test.

* fix(cli): fourth-round review on the Java/JVM path rule

* fix(cli): fifth-round review on the Java/JVM path rule

* fix(cli): sixth-round review on the Java/JVM path rule

* fix(cli): seventh-round review on the Java/JVM path rule

---------

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: qwen-code-ci-bot <qwen-code-ci-bot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant