Skip to content

running.md

Latest commit

 

History

History
240 lines (186 loc) · 7.98 KB

running.md

File metadata and controls

240 lines (186 loc) · 7.98 KB
Raw

Running STREAMLINE

STREAMLINE can be run through:

  • Google Colab
  • Local Jupyter notebook
  • Config-driven full pipeline runs
  • Phase-by-phase command-line calls

For most command-line work, start with the .cfg runner. It keeps shared settings, phase toggles, and phase-specific parameters in one editable file.

Choose A Run Path

Use case Recommended path
Conference tutorial or first demo Google Colab notebook
Interactive local exploration STREAMLINE_Notebook.ipynb
Reproducible full pipeline run python run.py -c run_configs/local/<config>.cfg
Debugging one phase Phase CLI command
Faster local execution without Dask run_cluster = Parallel
Local Dask execution run_cluster = Local
HPC execution run_configs/hpc/<config>.cfg with BashSLURM, BashLSF, or a site-specific Dask cluster setting

Google Colab

Open the Colab notebook:

Open the STREAMLINE Colab notebook

At the top of the notebook, set the demo/run parameters. The notebook supports binary, multiclass, regression, and custom data modes.

Local Jupyter

From the repository root:

conda activate streamline
jupyter notebook

Open STREAMLINE_Notebook.ipynb. The top parameter block controls which demo or custom dataset is run.

Config-Driven Runs

Dry-run a config first:

python run.py -c run_configs/local/uci_binary_hcc.cfg --dry_run

Run a full binary demo:

python run.py -c run_configs/local/uci_binary_hcc.cfg

Run the multiclass and regression demos:

python run.py -c run_configs/local/uci_multiclass_student.cfg
python run.py -c run_configs/local/uci_regression_auto_mpg.cfg

The included configs are designed as reproducible examples. For a short notebook or Colab demonstration, use the notebook parameter block, which uses a smaller modeling budget and shows plots by default.

Partial-run examples:

python run.py -c run_configs/local/uci_binary_hcc.cfg --start_at p4
python run.py -c run_configs/local/uci_binary_hcc.cfg --stop_after p8
python run.py -c run_configs/local/uci_binary_hcc.cfg --only p6,p8,p11
python run.py -c run_configs/local/uci_binary_hcc.cfg --skip p3,p4

Config File Layout

Each config uses sections like:

[run]
output_path = out
experiment_name = UCIHCCPipeline
outcome_label = Class
outcome_type = Binary
instance_label = InstanceID
n_splits = 3
run_cluster = Serial
random_state = 42

[phases]
phase_order = p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11
do_p1 = True
do_p2 = True
do_p3 = True
do_p4 = True
do_p5 = True
do_p6 = True
do_p7 = True
do_p8 = True
do_p9 = True
do_p10 = True
do_p11 = True

[p6]
outcome_type = Binary
models = NB,LR,DT
model_params_json = None
scoring_metric = balanced_accuracy
metric_direction = maximize
n_trials = 200
timeout = 900

Use run_cluster = Local for a local Dask cluster, or run_cluster = Parallel for local joblib parallelism without Dask. Use scheduler configs for cluster runs rather than changing a local demo config in place.

Use model_params_json when a model needs specific Phase 6 wrapper settings, such as HEROS pop_size or ExSTraCS N. See Model Parameter JSON for examples.

For run_cluster = BashSLURM or run_cluster = BashLSF, the config runner submits scheduler jobs and then waits for STREAMLINE completion markers before starting the next phase. This keeps full config runs from starting P2 before P1 has finished writing CVDatasets. The wait can be adjusted with wait_for_cluster_completion, cluster_phase_timeout, and cluster_phase_poll_interval in the [run] section.

For long HPC runs, launch the config runner inside tmux or screen so a lost SSH connection does not stop the orchestration process. See HPC and Cluster Runs for Cedars SLURM and UPenn LSF templates, scheduler monitoring commands, and recovery patterns.

Parallel and Dask-backed runs show progress when tqdm/Dask progress support is available. Set STREAMLINE_PROGRESS=0 to disable these progress displays.

Use the included configs as templates. Local demo configs live in run_configs/local/, and scheduler/HPC templates live in run_configs/hpc/. The original top-level demo config paths are kept for backward compatibility.

  • run_configs/local/uci_binary_hcc.cfg
  • run_configs/local/uci_multiclass_student.cfg
  • run_configs/local/uci_regression_auto_mpg.cfg
  • run_configs/hpc/cedars_slurm_hcc.cfg
  • run_configs/hpc/upenn_lsf_hcc.cfg

Rerunning Phases

STREAMLINE stores resolved phase arguments in run_commands.pickle inside the experiment folder. If you rerun a phase and omit an option, the saved value can be reused. Explicit command-line or config values override the saved value.

Use this control when you need a clean rerun:

python -m streamline.p6_modeling.p6_cli \
  --output_path out \
  --experiment_name DemoBinary \
  --outcome_type Binary \
  --ignore_saved_run_command

Use --no_update_saved_run_command when you want to run with temporary settings without changing the saved command summary.

Phase CLI Commands

Each phase can also be run independently. A binary classification example:

python -m streamline.p1_data_process.p1_cli \
  --data_path data/UCIBinaryClassification \
  --output_path out \
  --experiment_name DemoBinary \
  --outcome_label Class \
  --outcome_type Binary \
  --instance_label InstanceID \
  --categorical_features data/UCIFeatureTypes/hcc_survival_categorical_features.csv \
  --quantitative_features data/UCIFeatureTypes/hcc_survival_quantitative_features.csv \
  --n_splits 3 \
  --force true

python -m streamline.p2_impute_scale.p2_cli --output_path out --experiment_name DemoBinary
python -m streamline.p3_feature_learning.p3_cli --output_path out --experiment_name DemoBinary
python -m streamline.p4_feature_importance.p4_cli --output_path out --experiment_name DemoBinary
python -m streamline.p5_feature_selection.p5_cli --output_path out --experiment_name DemoBinary

python -m streamline.p6_modeling.p6_cli \
  --output_path out \
  --experiment_name DemoBinary \
  --outcome_label Class \
  --outcome_type Binary \
  --instance_label InstanceID \
  --models NB,LR,DT \
  --scoring_metric balanced_accuracy \
  --metric_direction maximize

python -m streamline.p7_ensembles.p7_cli --output_path out --experiment_name DemoBinary
python -m streamline.p8_summary_statistics.p8_cli --output_path out --experiment_name DemoBinary
python -m streamline.p9_compare_datasets.p9_cli --output_path out --experiment_name DemoBinary

python -m streamline.p10_replication.p10_cli \
  --rep_data_path data/UCIRepBinaryClassification \
  --dataset_for_rep data/UCIBinaryClassification/hcc_survival.csv \
  --output_path out \
  --experiment_name DemoBinary

python -m streamline.p11_reporting.p11_cli \
  --experiment_path out/DemoBinary \
  --report_mode standard

python -m streamline.p11_reporting.p11_cli \
  --experiment_path out/DemoBinary \
  --report_mode replication

By default, P6 reruns the requested model/CV jobs and overwrites existing model artifacts. Add --skip_completed_models 1 when you want recovery behavior that skips completed jobsCompleted/job_model_* markers and runs only failed or missing model jobs.

More examples are available in sample_runcommands.txt.

Discovery Commands

Several phases can list registry options:

python -m streamline.p2_impute_scale.p2_cli --output_path out --experiment_name DemoBinary --list-imputers
python -m streamline.p2_impute_scale.p2_cli --output_path out --experiment_name DemoBinary --list-scalers
python -m streamline.p3_feature_learning.p3_cli --output_path out --experiment_name DemoBinary --list-learners
python -m streamline.p4_feature_importance.p4_cli --output_path out --experiment_name DemoBinary --list-models
python -m streamline.p6_modeling.p6_cli --output_path out --experiment_name DemoBinary --outcome_type Binary --list_models
python -m streamline.p7_ensembles.p7_cli --output_path out --experiment_name DemoBinary --list_ensembles