Add Voice Agents to Foundry Python SDK (generated code + REST)

[xitzhang (xitzhang)]

Summary

Adds Voice Agents, a new agent kind for speech-to-speech conversational AI, unified with the rest of the Agents API. This PR is part 1 of 2: generated code plus the REST/CRUD hand-written support. Part 2 (hand-written realtime WebSocket streaming client) is a follow-up PR based on this branch: #48957.

This is a split of #48939 into two independently reviewable pieces, at the requester's request, so the generated/REST surface and the hand-written streaming client can be reviewed separately. #48939 itself is untouched.

What's included

• Voice agent definition as a new kind="voice" on AgentDefinition: model, audio input/output, turn detection, greeting, tools, and avatar configuration. Managed like any other agent through project_client.agents (create_version, get, list, disable/enable, delete), with guided authoring via generate_agent.
• The agent_endpoint_conversations operation group for reading back persisted voice-agent conversation transcripts and audio.
• Telephony: bindings so a voice agent can receive calls through Teams Phone or Twilio, plus outbound telephony call jobs and campaigns through the new agent_telephony operation group.
• WebRTC transport, sub-agent consultation, and conversation-engine delegation on the voice agent definition.
• Samples under samples/agents/voice/ for the management lifecycle, guided authoring, richer definitions, versioning, and reading back persisted conversations/audio.
• Recorded tests for CRUD, conversations, and telephony; mocked tests for the Foundry-Features opt-in header.

Validation

• black: clean
• pylint: 10.00/10
• mypy: clean (only a pre-existing, unrelated issue in sample_workflow_multi_agent.py)
• Full test suite: 1466 passed, 117 skipped, 0 failed

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Changes recommended

Part-one regeneration and live tests currently depend on the deferred realtime client, while the new sample installation commands do not select the beta package.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds preview Voice Agent REST/CRUD, conversation, telephony, samples, tests, and supporting generated API metadata to azure-ai-projects.

Changes:

• Adds Voice Agent models and operation groups.
• Adds management, conversation, and telephony samples/tests.
• Introduces beta release metadata and live-test resources.

File summaries

File	Description
tsp-location.yaml.saved	Updates TypeSpec source commit.
tests/test_base.py	Adds sanitized voice model setting.
tests/samples/test_samples.py	Adds sync voice sample coverage.
tests/samples/test_samples_async.py	Adds async voice sample coverage.
tests/samples/llm_instructions.py	Reformats validation instructions.
tests/foundry_features_header/test_foundry_features_header_on_ga_operations.py	Tests header casing.
tests/foundry_features_header/test_foundry_features_header_on_ga_operations_async.py	Tests async header casing.
tests/foundry_features_header/test_agent_telephony_protocol.py	Tests sync telephony requests.
tests/foundry_features_header/test_agent_telephony_protocol_async.py	Tests async telephony requests.
tests/foundry_features_header/foundry_features_header_test_base.py	Adds Voice Agent header cases.
tests/conftest.py	Adjusts recording sanitizers.
tests/agents/test_voice_agent_telephony.py	Adds skipped sync telephony tests.
tests/agents/test_voice_agent_telephony_campaign.py	Adds skipped campaign tests.
tests/agents/test_voice_agent_telephony_campaign_async.py	Adds skipped async campaign tests.
tests/agents/test_voice_agent_telephony_async.py	Adds skipped async telephony tests.
tests/agents/test_voice_agent_crud.py	Tests sync Voice Agent CRUD.
tests/agents/test_voice_agent_crud_async.py	Tests async Voice Agent CRUD.
tests/agents/test_voice_agent_conversations.py	Tests recorded conversation reads.
tests/agents/test_voice_agent_conversations_async.py	Tests async conversation reads.
tests.yml	Adds live-test pipeline.
test-resources.bicep	Provisions Foundry test resources.
test-resources-post.ps1	Deploys the realtime test model.
samples/hosted_agents/sample_session_log_stream.py	Explicitly enables streaming.
samples/hosted_agents/sample_session_log_stream_async.py	Enables async streaming.
samples/agents/voice/sample_voice_agent_with_tools.py	Demonstrates Voice Agent tools.
samples/agents/voice/sample_voice_agent_versions.py	Demonstrates version management.
samples/agents/voice/sample_voice_agent_read_conversation.py	Demonstrates transcript reads.
samples/agents/voice/sample_voice_agent_read_conversation_audio.py	Demonstrates audio reads.
samples/agents/voice/sample_voice_agent_generate.py	Demonstrates guided authoring.
samples/agents/voice/sample_voice_agent_basic.py	Demonstrates sync lifecycle.
samples/agents/voice/sample_voice_agent_basic_async.py	Demonstrates async lifecycle.
samples/agents/sample_workflow_multi_agent_with_mcp_approval.py	Updates type suppressions.
samples/agents/sample_workflow_multi_agent_async.py	Updates async type suppressions.
README.md	Lists Voice Agent samples.
pyproject.toml	Marks release as beta.
PostEmitter.ps1	Adds generation customizations.
docs/public-methods.md	Documents new operations.
CHANGELOG.md	Describes the preview release.
operations/_patch.py	Exports conversation customization.
operations/_patch_evaluators.py	Updates generated JSON typing.
operations/_patch_evaluation_rules.py	Adds type suppressions.
operations/_patch_datasets.py	Updates generated JSON typing.
operations/__init__.py	Exports new operation groups.
models/_patch.py	Adds Voice feature headers and model fix.
aio/operations/_patch.py	Exports async conversation customization.
aio/operations/_patch_evaluators_async.py	Updates async evaluator typing.
aio/operations/_patch_evaluation_rules_async.py	Adds async type suppressions.
aio/operations/_patch_datasets_async.py	Updates async dataset typing.
aio/operations/__init__.py	Exports async operation groups.
aio/_patch.pyi	Updates async stubs.
aio/_patch.py	Adds headers and decompression workaround.
aio/_client.py	Wires async operation groups.
_version.py	Sets version 2.7.0b1.
_utils/utils.py	Adds ETag helpers.
_unions.py	Adds Voice Agent unions.
_patch.pyi	Updates sync stubs.
_patch.py	Adds telephony preview headers.
_client.py	Wires sync operation groups.
assets.json	Updates test asset tag.
api.metadata.yml	Updates API snapshot metadata.
.env.template	Documents Voice Agent variables.

Review details

• Files reviewed: 68/72 changed files
• Comments generated: 11
• Review effort level: Balanced

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

[Pilot] PR Pipeline Failure Analysis

What failed

Build 6809773 (python - pullrequest) failed on all 6 platforms (Ubuntu2404_313, Ubuntu2404_314, macos311, ubuntu2404_310, ubuntu2404_310_coverage, windows2022_312) plus the Build Analyze job, for sdk/ai/azure-ai-projects. Three distinct, unrelated root causes were found:

1. Spelling check (Build Analyze job) — validation failure, fixable
cspell failed with ~123 unknown-word errors across CHANGELOG.md, api.md, and the generated/hand-written voice-agent source (models/_models.py, models/_enums.py, operations/_operations.py, _patch_agent_endpoint_conversations*.py, samples). New domain terms introduced by this PR aren't in the cspell dictionary: PSTN/pstn, PCMA/Pcma/pcma, PCMU/Pcmu/pcmu, BYOM, BYOS, MCPHTTP, DTMF/dtmf, retriable.

2. Test-proxy playback failures — 4,584 failures (~382 unique tests × 6 platforms × 2 wheel/sdist variants), infrastructure/recording pattern
Nearly all pre-existing recorded tests (e.g. test_ai_agents_instrumentor, test_telemetry, test_agent_session_files_crud, test_samples_evaluations, etc.) fail identically with:

azure.core.exceptions.HttpResponseError: Operation returned an invalid status 'None'

raised from devtools_testutils/proxy_testcase.py:117 inside start_record_or_playback, meaning the test-proxy playback request itself got no/invalid HTTP response. This affects the entire existing recorded-test corpus uniformly across every platform/package-variant combination in the same build — consistent with a test-proxy/tooling-level issue rather than per-test logic bugs, but it is unusually broad (nearly all recorded tests) for a typical infra blip and should be confirmed against a rerun before ruling out an environment regression from this PR's .venv_sdist/.venv_whl or proxy configuration changes.

3. New voice-agent test failures — 48 + 24 failures, code-fixable
In sdk/ai/azure-ai-projects/tests/agents/telemetry/test_responses_instrumentor_raw_response.py (TestRawResponseStreaming), new tests added by this PR fail with two related errors:

• AttributeError: 'LegacyAPIResponse' object has no attribute 'close' (lines 208, 226, 347, 361) — raw.close()/await raw.close() is called on an OpenAI LegacyAPIResponse object, which doesn't expose close() in the installed openai SDK version.
• AssertionError: Expected at least 1 span, got 0 (lines 168, 317) — test_async_with_raw_response_streaming_produces_telemetry / test_sync_with_raw_response_streaming_produces_telemetry expect the responses instrumentor to emit a span for raw-response streaming, but none is captured.

Relevant pipeline output

##[error]sdk/ai/azure-ai-projects/CHANGELOG.md:14:680 - Unknown word (PSTN)
##[error]sdk/ai/azure-ai-projects/api.md:9206:36 - Unknown word (PSTN)
##[error]sdk/ai/azure-ai-projects/azure/ai/projects/models/_enums.py:1185:24 - Unknown word (DTMF)
##[error]Spelling errors detected. To correct false positives or learn about spell checking see: https://aka.ms/azsdk/engsys/spellcheck
##[error]PowerShell exited with code '1'.

tests/agents/telemetry/test_responses_instrumentor_raw_response.py:208: AttributeError
E       AttributeError: 'LegacyAPIResponse' object has no attribute 'close'

tests/agents/telemetry/test_responses_instrumentor_raw_response.py:168: AssertionError
E       AssertionError: Expected at least 1 span, got 0

azure.core.exceptions.HttpResponseError: Operation returned an invalid status 'None'
../../../.venv/azure-ai-projects/.venv_sdist/lib/python3.13/site-packages/devtools_testutils/proxy_testcase.py:117: HttpResponseError

Recommended next steps

• Add the flagged domain-specific words (PSTN, PCMA, PCMU, BYOM, BYOS, MCPHTTP, DTMF, retriable, and their case variants) to the cspell custom dictionary/config for sdk/ai/azure-ai-projects.
• Fix test_responses_instrumentor_raw_response.py: replace raw.close()/await raw.close() with the correct close API for the installed openai LegacyAPIResponse type (e.g. raw.response.close() or consuming the underlying httpx response), and investigate why the responses instrumentor emits no span for raw-response streaming so the two produces_telemetry assertions pass.
• Re-run the pipeline once the above are fixed to confirm whether the mass test-proxy playback failures (invalid status 'None') persist; if they do, escalate as a test-proxy/tooling infrastructure issue since the failure is uniform across nearly all recorded tests and all 6 platforms rather than isolated to specific new tests.
• See https://aka.ms/ci-fix

Automated fix: Requested

Generated by Pipeline Analysis Next Steps · auto · 182 AIC · ⌖ 7.59 AIC · ⊞ 9.2K · ◷