Update only when different for several harvester scripts#603
Open
ChrisJChang wants to merge 3 commits into
Open
Update only when different for several harvester scripts#603ChrisJChang wants to merge 3 commits into
ChrisJChang wants to merge 3 commits into
Conversation
Write ColliderBit_models_rollcall.hpp and Py8Collider_typedefs.hpp to candidate files and install them with update_only_if_different(), as the other harvesters already do. Previously these two headers were rewritten unconditionally (with a fresh timestamp) every time the harvester ran, which dirtied ColliderBit_rollcall.hpp/ColliderBit_types.hpp and forced a large ColliderBit and Core rebuild after every reconfiguration even when nothing had changed. https://claude.ai/code/session_01QaXhcsFuD88dDh7Qvq1pSA
The harvester custom commands (and the standalone functor facilitator) depended on CMakeCache.txt, so every rerun of cmake caused all of them to run again at the next make. Replace that dependency with a stamp file recording the full harvester command line, updated at configure time via copy_if_different. The harvesters still rerun when their inputs, the -Ditch list, or the Python interpreter change, but no longer after a no-op reconfigure. https://claude.ai/code/session_01QaXhcsFuD88dDh7Qvq1pSA
Two remaining causes of mass rebuilds after a cmake rerun: 1. backend_harvester.py, model_harvester.py, particle_harvester.py and make_module_rollcall() in harvesting_tools.py wrote their outputs unconditionally, dirtying backend_rollcall.hpp, backend_types_rollcall.hpp, model_rollcall.hpp, model_types_rollcall.hpp, particle_database.cpp and module_rollcall.hpp on every harvester run. Route them all through update_only_if_different(), as module_harvester.py already does. Since backend and model types are included by nearly every GAMBIT source file, this alone forced a near-full rebuild whenever the harvesters re-ran. 2. The harvesters emit include lists and template instantiations by iterating over Python sets, whose iteration order changes between interpreter invocations due to hash randomisation. This made the generated files differ from run to run, defeating update_only_if_different() even where it was already used. Sort all set-based collections at the point of emission so the generated content is deterministic. With this, rerunning the harvesters (e.g. after any cmake reconfiguration, which deletes their stamp files) leaves all twelve generated files untouched unless their content genuinely changes. https://claude.ai/code/session_01QaXhcsFuD88dDh7Qvq1pSA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR writes a few scratch build files for the results of some of the harvester scripts, and checks whether they are different before updating the files. The purpose of this is that when rerunning cmake, we want to avoid rebuilding files that aren't actually any different.
My test for this was running a build, running cmake .. (with no code changes at all), and rerunning make again to see what still builds. This reduces it quite a bit, but there are still a few files that rebuild. There are few enough that I do not think it is too much of a problem. These are SpecBit files that are being built because there is a FlexibleSUSY configure script that is generating some config header (only included in SpecBit files). I can try to solve that in another PR if desired.