[JAX] Fix undefined sr_rng_state that breaks --dry-run in two encoder examples - #3498
Open
Anai-Guo wants to merge 1 commit into
Open
[JAX] Fix undefined sr_rng_state that breaks --dry-run in two encoder examples#3498Anai-Guo wants to merge 1 commit into
Anai-Guo wants to merge 1 commit into
Conversation
test_model_parallel_encoder.py and test_multiprocessing_encoder.py build the dry-run rngs dict from `sr_rng_state`, which is never bound in train_and_evaluate; the local is `sr_rng`. Running either example with --dry-run raises NameError before the single train step. test_single_gpu_encoder.py and test_multigpu_encoder.py already use `sr_rng` in the same spot. Signed-off-by: Anai Guo <antai12232931@outlook.com>
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
examples/jax/encoder/test_model_parallel_encoder.pyandtest_multiprocessing_encoder.pybuild the RNG dict for the--dry-runpathfrom a name that is never bound:
The local inside
train_and_evaluateissr_rng(bound a few dozen linesearlier alongside
params_rng/dropout_rng), andsr_rng_stateis a namefrom the quantizer API (
transformer_engine/jax/cpp_extensions/quantization.py),not from these scripts. So
both die with
NameError: name 'sr_rng_state' is not definedbefore the singletrain step ever runs.
The other two examples already do it correctly
The same block in the two sibling examples uses
sr_rng:test_single_gpu_encoder.pyrngs = {DROPOUT_KEY: dropout_rng, SR_KEY: sr_rng}test_multigpu_encoder.pyrngs = {DROPOUT_KEY: dropout_rng, SR_KEY: sr_rng}test_model_parallel_encoder.py... SR_KEY: sr_rng_state}←test_multiprocessing_encoder.py... SR_KEY: sr_rng_state}←The non-dry-run loop in the two broken files also uses
sr_rng(lines 395 and 491), so
sr_rngis unambiguously the intended name.Fix
sr_rng_state→sr_rngin the two--dry-runbranches. Two lines.🤖 Generated with Claude Code