Migrate AzureBatchHook/Operator to azure-batch 15.x (track 2 SDK)

[arieleli01212]

Description

The azure-batch package underwent a complete API overhaul in version 15.x, moving from the old Azure SDK track 1 (BatchServiceClient, batch_auth) to track 2 (BatchClient, azure-identity). This PR updates AzureBatchHook and AzureBatchOperator to work with the new SDK and removes the <15.0.0 upper bound cap that was explicitly blocking users from upgrading.

Changes

hooks/batch.py

• Replace BatchServiceClient with BatchClient(endpoint, credential)
• Replace batch_auth.SharedKeyCredentials with AzureNamedKeyCredential
• Catch ResourceExistsError (from azure.core.exceptions) instead of BatchErrorException
• Rename model types: PoolAddParameter → BatchPoolCreateOptions, JobAddParameter → BatchJobCreateOptions, TaskAddParameter → BatchTaskCreateOptions, ImageReference → BatchVmImageReference, PoolInformation → BatchPoolInfo
• Update enum references to use SCREAMING_SNAKE_CASE values: BatchNodeState.IDLE, BatchTaskState.COMPLETED, BatchTaskExecutionResult.FAILURE
• Replace the nested sub-client calls (client.pool.add, client.compute_node.list, etc.) with the new flat API (client.create_pool, client.list_nodes, etc.)
• Use LRO methods begin_delete_pool, begin_delete_job, begin_terminate_job
• Remove CloudServiceConfiguration support — Azure has retired cloud-service VM pools

operators/batch.py

• Update type hints for batch_job_manager_task, batch_job_preparation_task, etc. to use the new Batch-prefixed names
• Remove os_family and os_version parameters (no longer applicable without CloudServiceConfiguration)
• Update node-state set to use BatchNodeState.START_TASK_FAILED / UNUSABLE / IDLE
• Update on_kill() and clean_up() to use the LRO methods

pyproject.toml

• Lift the azure-batch constraint from >=8.0.0,<15.0.0 to >=15.0.0

Tests

• Rewrite hook and operator tests to use the new BatchClient, model types, and enum values

Relates

Fixes the blocking comment left in pyproject.toml about needing a hook rewrite before lifting the <15.0.0 cap.

closes: #66466

Testing

• I ran pytest providers/microsoft/azure/tests/unit/microsoft/azure/hooks/test_batch.py and all tests pass
• I ran pytest providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_batch.py and all tests pass

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack