[BugFix] Avoid unnecessary SAC and CQL collector spec queries

[aswanth-07]

Description

SACTrainer and CQLTrainer now query the collector environment for an action spec only when automatic target entropy cannot be resolved from the loss or actor.

Numeric target_entropy values and actor-provided action specs no longer require collector.getattr_env. SAC's existing collector fallback remains unchanged when neither the loss nor actor supplies the required spec.

The regression constructs both trainers through their public APIs with a collector that has no environment accessor. It covers explicit zero target entropy and automatic entropy derived from a two-dimensional actor spec, then verifies the resolved target. A separate case verifies that SAC still requests the collector spec exactly once and resolves the same two-dimensional target to -2.0.

Motivation and Context

Closes #4376.

This is the SAC/CQL follow-up requested during review of #4368. Collectors without getattr_env currently fail during trainer construction even when the loss already has enough information to determine target entropy.

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Tests

• python -m pytest test/test_trainer.py::TestEntropyTrainerActionSpec test/objectives/test_sac.py test/objectives/test_cql.py -q -k 'EntropyTrainerActionSpec or test_sac_target_entropy_auto or test_sac_target_entropy_explicit or test_cql_target_entropy_auto' (25 passed, 2894 deselected)
• Repository hooks for line endings, formatting, flake8, pydocstyle, pyupgrade, codespell, autoflake, and docstring placement passed on the three changed files.
• py -3.13 scripts/check-docstring-args torchrl/trainers/algorithms/sac.py torchrl/trainers/algorithms/cql.py passed.
• The full test suite was not run locally.

Checklist

• I have read the CONTRIBUTION guide (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.

AI assistance disclosure: AI-assisted development tools were used during investigation and implementation. The reported tests were run against the final diff.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/4377

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 16 Awaiting Approval

As of commit 65600d1 with merge base ee1f354 ():

AWAITING APPROVAL - The following workflows need approval before CI can run:

• Build Aarch64 Linux Wheels (gh)
• Build Linux Wheels (gh)
• Build M1 Wheels (gh)
• Build Windows Wheels (gh)
• Continuous Benchmark (PR) (gh)
• Examples Tests on Linux (gh)
• Generate documentation (gh)
• Habitat Tests on Linux (gh)
• Libs Tests on Linux (gh)
• Lint (gh)
• LLM Tests on Linux (gh)
• Push Binary Nightly (gh)
• SOTA Tests on Linux (gh)
• Tutorials Tests on Linux (gh)
• Unit-tests on Linux (gh)
• Validate Test Partitioning (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[aswanth-07]

Rebased this onto current main in 65600d1 and resolved the trainer test import overlap. The focused SAC/CQL trainer regression remains green: 25 passed, 2894 deselected. Changed-file formatting, lint, pydocstyle, pyupgrade, codespell, autoflake, and docstring-argument checks also pass.