fix(scheduler): 2gpu 2ppl overlap fix

[TianyeGGBond]

Problem

pending_bucket_gen is a per-cycle snapshot that disappears immediately after a GENERATION pending request is consumed by a signal. Between the signal being sent and the arrival of the first ProgressReport in begin_progress_batch(), the planner cannot see the demand for this pipeline and will skip it—causing 2gpu 2ppl task fail.

Solution

Introduce rollout_open_pipelines: Dict[str, Optional[int]] in SchedulerState:

• Set when request_gpus(priority=GENERATION) is enqueued (carrying step_target_estimate)

• Cleared when the GENERATION cluster is released or clear_progress() exits all batch streams

This covers the entire lifecycle from "pipeline initiating a rollout request" to "rollout completely ending,"

Changes

• state.py: Added the rollout_open_pipelines field

• scheduler.py: Added set/clear to the correct lifecycle nodes; removed the _clear_rollout_intent_locked() helper

• planner.py: Replaced the pending_bucket_gen parameter with rollout_open_pipelines;

Removed has_pending_generation_request / get_pending_generation_step_target_estimate

• tests: Updated all calls; incidentally fixed a deprecation warning for asyncio.get_event_loop().

One RLHF rollout round — signal lifecycle

  request_gpus()     signal consumed     begin_progress_batch()    end_progress_batch()    release GPU
        │                   │                      │                        │                   │
        ▼                   ▼                      ▼                        ▼                   ▼
────────●───────────────────●──────────────────────●────────────────────────●───────────────────●────▶
        │                   │                      │                        │
        │         pending request                  │
        │         removed from bucket              │
        │                   │                      │
        │◄───── gap ────────►                      │
        │   scheduler sees no demand               │
        │   pipeline skipped, GPU stolen           │
        │                   │                      │
        │                                          │
        │  latest_progress_by_pipeline ────────────●────────────────────────● cleared by clear_progress()
        │  (populated on first ProgressReport)     │
        │                                          │
        ├── pending_bucket_gen ──────●             │
        │   (old signal, gone        │             │
        │    after signal consumed)  ▲             │
        │                            └─ too short  │
        │                                          │
        └── rollout_open_pipelines ────────────────────────────────────────●─── cleared
            (new signal, set at                                             │
             request_gpus() time)                                  on release / clear_progress()


BEFORE fix:  planner eligibility = pending_bucket_gen OR active_dp_workers OR active_allocations
             └─ gap window: all three are empty → pipeline skipped

AFTER fix:   planner eligibility = rollout_open_pipelines OR active_dp_workers OR active_allocations
             └─ gap window: rollout_open_pipelines still set → pipeline stays eligible