Skip to content

[BUG] Custom operator error return values are inconsistent across SDK implementations #1874

@jonathannorris

Description

@jonathannorris

Summary

When custom operators (fractional, sem_ver, starts_with, ends_with) receive invalid input, the return value varies across SDK implementations. In JSONLogic, null/nil and false have different semantics — null typically causes a fallback to the default variant, while false takes the false branch of a conditional. This means the same malformed targeting rule produces different evaluation outcomes depending on which SDK evaluates it.

The Problem

sem_vernull vs false on error

The most impactful inconsistency. When sem_ver fails to parse a version or receives an unknown operator:

SDK Error return JSONLogic effect
Go false Takes the false branch
Java null Triggers default variant fallback
JS/TS false Takes the false branch
C#/.NET false Takes the false branch
Python None Triggers default variant fallback
Rust Null Triggers default variant fallback

In a rule like {"if": [{"sem_ver": [...]}, "variant-a", "variant-b"]}, a parse failure returns "variant-b" in Go/JS/.NET but triggers the default variant in Java/Python/Rust.

starts_with / ends_with — Go internal inconsistency

Go returns different values for two operators that share the same input validation:

Operator Go error return
starts_with nil
ends_with false

All other SDKs are internally consistent for these two operators (Java returns null for both; JS/.NET return false for both; Python returns None or False depending on the error type).

fractional — divergent "no bucket matched" behavior

When a user's hash doesn't fall into any bucket range (e.g., due to weight misconfiguration):

SDK No-match return Effect
Go "" (empty string) Variant lookup fails → error
Java throws JsonLogicEvaluationException Evaluation error propagated
JS/TS null Default variant fallback
C#/.NET "" (empty string) Variant lookup fails → error
Python None Default variant fallback
Rust Value::Null Default variant fallback

Three different behaviors: empty string (Go/.NET), exception (Java), or null fallback (JS/Python/Rust).

Full Error Return Matrix

Operator Error Type Go Java JS/TS .NET Python Rust
fractional Parse/input error nil null null null None Null
fractional No bucket matched "" throws null "" None Null
sem_ver Any error false null false false None Null
starts_with Any error nil null false false None/False N/A (built-in)
ends_with Any error false null false false None/False N/A (built-in)

Suggested Fix

  1. Standardize error returns per operator and document the expected behavior in the spec:
    • fractional errors → null (triggers default variant fallback)
    • sem_ver errors → decide between false or null and apply consistently
    • starts_with / ends_with errors → decide between false or null and apply consistently
  2. Fix Go's starts_with/ends_with inconsistency — both should return the same value on error.
  3. Fix fractional no-bucket-match — standardize to null (fallback to default) rather than empty string or exception.
  4. Add Gherkin test cases that exercise error paths (wrong arg count, invalid types, unparseable versions) and assert the expected return value.

Related: #1770

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions