Summarise the values a field actually has - #124
Merged
Conversation
summarize_array called plain min/max/mean, which propagate NaN, so one missing value emptied all three statistics. Measured on a temporal mean of a field masked over half its faces: three faces held finite means and the summary reported min, max and mean all NaN. Land masks are ordinary here, so that was most fields. The same payloads were not valid JSON either. json.dumps writes NaN as the bare token NaN, which no parser is required to accept, so a strict client rejected the whole result rather than the one number. Statistics now skip non-finite entries, report None when nothing is finite, and carry n_finite/n_total only when the two differ, so a complete field costs no extra bytes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
summarize_arrayused plainmin/max/mean, which propagate NaN, so a field masked over half its faces reportedmin,maxandmeanall NaN while three faces held finite values. The same payload was also not valid JSON, sincejson.dumpswrites NaN as a bareNaNtoken that a strict client rejects. Statistics now skip non-finite entries, reportNonewhen nothing is finite, and addn_finite/n_totalonly when they differ; 880 passed, 18 skipped locally with pre-commit green.