Let the other simulators run when the first one fails - #311
Merged
Conversation
An FMI job that runs several simulators over one FMU (#297, #309) gives the first of them a veto over the rest. When it fails, the TimeoutError handler calls writeResultAndExit, the process is gone, and the loop that runs fmisimulators[1:] never starts -- but a row is still written for every simulator, recording the phase the first one stopped at. The other tools are marked as having failed at a model they were never given. checkOutputTimeout raises TimeoutError for a command that exits non-zero as well as for one that runs out of time, so this covers an ordinary failure and not just a slow one. In the master-fmi run of 2026-08-12 this cost 433 models. They had never simulated under OMSimulator -- phase 5 in every master-fmi run back to 2026-08-08 -- but FMPy ran them, and the previous master-fmi-fmpy run has them at phase 6 or 7. In the first combined run they are phase 5 in both tables, with FMPy's simulation time recorded as exactly 0: OMSimulator failed in under a second for 344 of them, and FMPy never started. They show up as 433 of the 653 regressions on the comparison page, next to the 213 that PR #309 really did find. It is also why that page reports OMSimulator's version, and its JSON parse warning, in place of "FMPy version 0.3.29": FMPy never ran to say what it was. The first simulator now records its failure and falls through to the others, which is what every simulator after it already did. Nothing changes when only one simulator is configured: that case still ends the model where it always has, so the non-FMI paths and the single-simulator FMI jobs are untouched. Verified on MyLibrary.Blocks.Examples.PID_Controller with the first simulator replaced by /bin/false: before phase 5, "simulators": {} after phase 5, "simulators": {"fmpy": {"sim": 0.64, "phase": 7}} and, unchanged in both: one failing simulator phase 5, os._exit as before both simulators working OMSimulator phase 7, fmpy phase 7 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MYvMaAotMy425H7nvKzWX7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An FMI job that runs several simulators over one FMU (#297, #309) gives the first of
them a veto over the rest. When it fails, the
TimeoutErrorhandler callswriteResultAndExit, the process is gone, and the loop that runsfmisimulators[1:]never starts — but a row is still written for every simulator, recording the phase the
first one stopped at. The other tools are marked as having failed at a model they were
never given.
checkOutputTimeoutraisesTimeoutErrorfor a command that exits non-zero as well asfor one that runs out of time, so this is an ordinary failure and not just a slow one.
What it cost
In the first combined
master-fmirun (2026-08-12 11:04 UTC), 433 models.They had never simulated under OMSimulator — phase 5 in every
master-fmirun back to2026-08-08 — but FMPy ran them, and the previous
master-fmi-fmpyrun has them at phase 6or 7. In the combined run they are phase 5 in both tables, and FMPy's simulation time is
recorded as exactly 0: OMSimulator failed in under a second for 344 of them, and FMPy
never started.
master-fmi-fmpyregressions, 08-11 06:50 → 08-12 11:04Of the 440 regressions outside the #309 libraries, 436 report exactly OMSimulator's
finalphase, and their combined FMPy simulation time is 0.0 minutes againstOMSimulator's 78.2.
It is also why the comparison page reports OMSimulator's version — and its JSON parse
warning — where
FMPy version 0.3.29used to be:FMPy never ran to say what it was. Both sides of that comparison are the same omc commit
7fd853cf, so none of this is a compiler change.The change
The first simulator now records its failure and falls through to the others — which is
what every simulator after it already did:
Nothing changes when only one simulator is configured: that case still ends the model
where it always has, so the non-FMI code paths and the single-simulator FMI jobs are
untouched.
Testing
On
MyLibrary.Blocks.Examples.PID_Controller, with the first simulator replaced by/bin/false:phasesimulators{}{"fmpy": {"sim": 0.64, "diff": {...}, "phase": 7}}and unchanged in both:
os._exitas beforeNote
Same shape as #308/#310: a failure path that leaves before the accounting is done. Worth
watching for others.
generated by Claude Code