Skip to content

Bump Python version to 3.13#3401

Open
ndonyapour wants to merge 1 commit intoflyteorg:masterfrom
ndonyapour:feature/pyversion-bump
Open

Bump Python version to 3.13#3401
ndonyapour wants to merge 1 commit intoflyteorg:masterfrom
ndonyapour:feature/pyversion-bump

Conversation

@ndonyapour
Copy link

Tracking issue

Closes flyteorg/flyte#6993

Why are the changes needed?

Python 3.13 was released in October 2024, and flytekit should support it. The previous requires-python constraint was >=3.10,<3.13, which explicitly excluded Python 3.13. Several issues needed to be addressed:

  1. The requires-python constraint needed to be updated from >=3.10,<3.13 to >=3.10,<3.14 to allow Python 3.13
  2. Python 3.13 was missing from the project classifiers in pyproject.toml
  3. The PythonVersion enum didn't include Python 3.13, causing ValueError: (3, 13) is not a valid PythonVersion errors when running tests
  4. CI workflows didn't test Python 3.13
  5. Plugin default images didn't support Python 3.13
  6. Tests for tensorflow/pytorch were failing on Python 3.13 because these libraries don't support Python 3.12+ yet

What changes were proposed in this pull request?

Core Changes

  • pyproject.toml:

    • Updated requires-python from ">=3.10,<3.13" to ">=3.10,<3.14" to allow Python 3.13
    • Added "Programming Language :: Python :: 3.13" to project classifiers
  • flytekit/configuration/default_images.py:

    • Added PYTHON_3_13 = (3, 13) to PythonVersion enum
    • Added Python 3.13 default image prefix: "cr.flyte.org/flyteorg/flytekit:py3.13-"

Plugin Updates

  • plugins/flytekit-sqlalchemy/flytekitplugins/sqlalchemy/task.py: Added Python 3.13 to SQLAlchemyDefaultImages._DEFAULT_IMAGE_PREFIXES
  • plugins/flytekit-openai/flytekitplugins/openai/batch/task.py: Added Python 3.13 to OpenAIBatchDefaultImages._DEFAULT_IMAGE_PREFIXES

Test Updates

Added pytest.importorskip() and pytestmark = pytest.mark.skipif(sys.version_info >= (3, 12), ...) to skip tensorflow/pytorch tests on Python 3.12+:

  • tests/flytekit/unit/extras/pytorch/test_checkpoint.py
  • tests/flytekit/unit/extras/pytorch/test_native.py
  • tests/flytekit/unit/extras/pytorch/test_transformations.py
  • tests/flytekit/unit/extras/tensorflow/record/test_transformations.py
  • tests/flytekit/unit/extras/tensorflow/record/test_record.py
  • tests/flytekit/unit/extras/tensorflow/model/test_model.py
  • tests/flytekit/unit/extras/tensorflow/model/test_transformations.py

These tests are automatically skipped on Python 3.12+ because tensorflow and pytorch don't support these Python versions yet.

CI/CD Updates

  • .github/workflows/pythonbuild.yml:

    • Added Python 3.13 to scheduled test runs: ["3.10","3.11","3.12","3.13"]
    • Updated extras test skip condition: if: ${{ matrix.python-version != '3.12' && matrix.python-version != '3.13' }}
    • Added Python 3.13 exclusions for incompatible plugins (airflow, mlflow, modin, vaex, ray, kf-pytorch, onnx-pytorch)
    • Added Python 3.13 exclusion for pandas<2.0.0 test matrix
  • .github/workflows/pythonpublish.yml:

    • Added Python 3.13 to Docker image build matrix: ["3.10","3.11","3.12","3.13"]
  • .github/workflows/build_image.yml:

    • Already includes Python 3.13 (no changes needed)

Documentation

  • dev-requirements.in: Updated comments to reflect that tensorflow/torch don't support Python 3.12+ yet

How was this patch tested?

Setup process

  1. Created a Python 3.13 uv environment:

    uv venv --python 3.13
    source .venv/bin/activate  # or `.venv\Scripts\activate` on Windows
  2. Installed dependencies and dev requirements:

uv pip install -r dev-requirements.in


3. Verified Python 3.13 enum works correctly:
```bash
python -c "from flytekit.configuration.default_images import PythonVersion; print(PythonVersion((3, 13)))"
# Output: PythonVersion.PYTHON_3_13
  1. Ran unit tests using make commands:

    make unit_test
  2. Verified tensorflow/pytorch tests are properly skipped:

    make unit_test_extras

Test Results

  • ✅ Python 3.13 enum correctly recognizes (3, 13)
  • ✅ Core unit tests pass on Python 3.13
  • ✅ Tensorflow/pytorch tests are automatically skipped on Python 3.13
  • ✅ CI workflows updated to test Python 3.13 in scheduled runs
  • ✅ Docker images will be built for Python 3.13 on releases

Screenshots

N/A (code changes only)

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

N/A

Docs link

N/A (no documentation changes required)


Note: Some plugins are excluded from Python 3.13 CI runs due to dependency incompatibilities (documented in workflow exclusions). These will be re-enabled as dependencies add Python 3.13 support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Python 3.13 support to Flytekit

1 participant