Azure · xitzhang (xitzhang) · Sep 10, 2026
@@ -23,7 +23,7 @@ AZURE_AI_PROJECTS_CONSOLE_LOGGING=
 FOUNDRY_PROJECT_ENDPOINT=
 FOUNDRY_PROJECT_API_KEY=
 FOUNDRY_MODEL_NAME=
-# Read by the recorded voice-agent CRUD and telephony tests
+# Read by the recorded voice-agent CRUD, conversation, realtime-live, and telephony tests
 # (tests/test_base.py and friends), not by any sample.
 FOUNDRY_VOICE_MODEL_NAME=
 # Read by the samples under samples/agents/voice/ (model deployment name, agent name, model type,

@@ -4,10 +4,11 @@
 
 ### Features Added
 
-* Added Voice Agents, a new agent kind for speech-to-speech conversational AI unified with the rest of the Agents API. Define a voice agent's model, audio, turn detection, greeting, and tools; manage it like any other agent; and reach it through telephony (inbound bindings or outbound calls/campaigns).
+* Added Voice Agents, a new agent kind for real-time, speech-to-speech conversational AI unified with the rest of the Agents API. Define a voice agent's model, audio, turn detection, greeting, and tools; manage it like any other agent; hold a live conversation with it over a WebSocket with barge-in and persisted conversation history/audio; and reach it through telephony (inbound bindings or outbound calls/campaigns).
 * The core voice agent definition, as a new `kind="voice"` on `AgentDefinition`:
   * Define a voice agent with `VoiceAgentDefinition`, configuring its model (`VoiceModelType`), audio input/output (`VoiceAgentAudioConfig`, `VoiceAgentAudioInputConfig`, `VoiceAgentAudioOutputConfig`), turn detection (`VoiceAgentTurnDetectionConfig` and its `VoiceAgentServerVadTurnDetection` / `VoiceAgentAzureSemanticVadTurnDetection` / `VoiceAgentAzureSemanticVadEnTurnDetection` / `VoiceAgentAzureSemanticVadMultilingualTurnDetection` variants), greeting (`VoiceAgentGreetingConfig` and its `VoiceAgentTemplateGreetingConfig` / `VoiceAgentLlmGeneratedGreetingConfig` variants), tools (`VoiceAgentTool`, `VoiceAgentFunctionTool`, `VoiceAgentMcpTool`, `VoiceAgentSystemTool` and its `VoiceAgentEndConversationSystemTool` variant, `VoiceAgentToolboxTool`), and avatar (`VoiceAgentAvatarConfig`). Manage it like any other agent through `project_client.agents` (`create_version`, `get`, `list`, `disable`/`enable`, `delete`).
   * Added guided authoring via `project_client.agents.generate_agent(GenerateVoiceAgentRequest(kind=AgentKind.VOICE, ...))`, which returns a service-generated starter definition that can be edited afterward through the standard `create_version`/`update` flow.
+  * Added a new `client.realtime` / `async_client.realtime` entry point for realtime speech-to-speech streaming. Use `with client.realtime.connect(agent_name=...) as connection:` to open a WebSocket connection, `connection.send(...)` to send strongly-typed client events (or use the `connection.response`, `connection.conversation.item`, and `connection.session` helpers), and iterate over `connection` to receive strongly-typed server events (`RealtimeServerEvent*`). Conversation items exchanged with `connection.conversation.item.create(...)` are `RealtimeConversationItemMessageSystem`, `RealtimeConversationItemMessageUser`, `RealtimeConversationItemMessageAssistant`, `RealtimeConversationItemFunctionCall`, `RealtimeConversationItemFunctionCallOutput`, `RealtimeMCPApprovalResponse`, or a raw `Mapping[str, Any]`. The new types `Realtime`, `RealtimeConnection`, and `RealtimeConnectionManager` (and their async equivalents `AsyncRealtime`, `AsyncRealtimeConnection`, `AsyncRealtimeConnectionManager`) are exported from `azure.ai.projects` / `azure.ai.projects.aio`. These WebSocket clients identify themselves to the service the same way the generated HTTP surface does, via a standard Azure SDK `User-Agent` header and an `x-ms-client-sdk` query parameter for paths where the header isn't forwarded, so service telemetry can attribute this traffic to the SDK; a caller-supplied `User-Agent` in `extra_headers` still takes precedence. Requires the optional `websockets` package for the sync client, or `aiohttp` for the async client.
   * Added the `agent_endpoint_conversations` operation group for reading back persisted voice-agent conversation transcripts and audio, for agents created with `store=True`.
   * Added the underlying `RealtimeConversationItem*`, `RealtimeMCP*`, `RealtimeResponseUsage`, and related realtime event/session models used by the voice agent WebSocket protocol.
 * Telephony, WebRTC, and sub-agent consultation:
@@ -18,13 +19,20 @@
   * Added sub-agent consultation, letting a voice agent consult sibling Foundry text agents as background specialists mid-conversation, through the new `subagent_config` property on `VoiceAgentDefinition` (`VoiceAgentSubagentConfig`, `VoiceAgentSubagent`, `VoiceAgentSubagentResponsePolicy`), and the new `session.subagent.started`/`session.subagent.completed`/`session.subagent.aborted` realtime server events.
   * Added an optional `conversation_engine` property on `VoiceAgentDefinition` (`VoiceConversationEngine`, `VoiceHostedAgentConversationEngine`) to delegate a voice agent's conversation handling to another hosted agent instead of configuring a model directly.
 
+### Dependency update
+
+* Added an optional dependency on `websockets` (sync `client.realtime`) and `aiohttp` (async `async_client.realtime`), required only when using the new voice agent realtime streaming APIs.
+
 ### Sample updates
 
 * Added voice agent samples under `samples/agents/voice/`:
   * `sample_voice_agent_basic.py` / `sample_voice_agent_basic_async.py` demonstrating the voice-agent management lifecycle: create, get, list, and delete.
   * `sample_voice_agent_generate.py` demonstrating guided authoring of a voice agent via `generate_agent` with `kind="voice"`.
   * `sample_voice_agent_with_tools.py` demonstrating a richer voice agent definition: audio configuration, turn detection, greeting, and tools.
   * `sample_voice_agent_versions.py` demonstrating voice-agent versioning: creating, drafting, listing, and publishing versions.
+  * `sample_voice_agent_live_text_conversation.py` / `sample_voice_agent_live_text_conversation_async.py` demonstrating a live, typed conversation with a voice agent over `client.realtime`/`async_client.realtime`.
+  * `sample_voice_agent_live_audio_conversation_async.py` demonstrating a hands-free, bidirectional live audio conversation over `async_client.realtime`.
+  * `sample_voice_agent_live_function_tool.py` demonstrating handling a client-executed function tool during a live voice-agent session.
   * `sample_voice_agent_read_conversation.py` demonstrating reading a persisted voice conversation's transcript back via `agent_endpoint_conversations`.
   * `sample_voice_agent_read_conversation_audio.py` demonstrating reading a persisted voice conversation's audio, both the merged whole-call recording and a single turn's segment, via `agent_endpoint_conversations`.
 * Added `sample_agent_web_iq.py` under `samples/agents/tools/`, demonstrating a Prompt Agent using the `WebIQPreviewTool`.

@@ -438,6 +438,34 @@ foreach ($f in $files) {
     Set-Content $f $c -NoNewline
 }
 
+# Regression guard: `_realtime.py` and `aio\_realtime.py` are hand-written files that are NOT
+# `_patch.py`-named, so they aren't covered by the emitter's own "never touch _patch.py" guarantee --
+# nothing in the TypeSpec emitter is aware these files exist. They carry the SDK client-identification
+# fix ported from the azure-ai-voicelive PR #48848 (a User-Agent header and x-ms-client-sdk query
+# parameter, both derived from `_USER_AGENT = UserAgentPolicy(sdk_moniker=...)`, with a case-insensitive
+# guard so a caller-supplied extra_headers User-Agent of any casing is honored instead of duplicated).
+# If a future `tsp-client update` ever starts generating (and thus silently overwriting) a file at either
+# of these paths, this fix would be lost with no other signal until someone happens to run the realtime
+# test suite. Fail the emit step immediately instead, right after regeneration, rather than relying on
+# that eventual test run.
+$realtimeFiles = @('azure\ai\projects\_realtime.py', 'azure\ai\projects\aio\_realtime.py')
+foreach ($f in $realtimeFiles) {
+    if (-not (Test-Path $f)) {
+        throw "PostEmitter safety check failed: '$f' is missing. This hand-written file (not tracked by the TypeSpec emitter) carries the SDK client-identification fix from PR #48848; if the emitter deleted or renamed it, restore it from git history before continuing."
+    }
+    $c = Get-Content $f -Raw
+    if ($c -notmatch 'UserAgentPolicy\(sdk_moniker=') {
+        throw "PostEmitter safety check failed: '$f' no longer defines _USER_AGENT via UserAgentPolicy(sdk_moniker=...). The SDK client-identification fix from PR #48848 appears to have been overwritten -- reinstate the User-Agent header + x-ms-client-sdk query param wiring."
+    }
+    if ($c -notmatch '_has_header_case_insensitive') {
+        throw "PostEmitter safety check failed: '$f' no longer guards the User-Agent header with _has_header_case_insensitive. A caller-supplied extra_headers User-Agent (in any casing) would be duplicated instead of honored -- reinstate the case-insensitive check."
+    }
+    if ($c -notmatch 'x-ms-client-sdk') {
+        throw "PostEmitter safety check failed: '$f' no longer sends the x-ms-client-sdk query parameter alongside the User-Agent header -- reinstate it so service telemetry can still attribute traffic on paths that don't forward the header."
+    }
+}
+Write-Host "PostEmitter safety check passed: SDK client-identification fix (PR #48848) is intact in both _realtime.py files."
+
 # Finishing by running 'black' tool to format code. 
 pip install black
 black --config ../../../eng/black-pyproject.toml .

@@ -2,6 +2,7 @@
 namespace azure.ai.projects
 
     class azure.ai.projects.AIProjectClient(AIProjectClientGenerated): implements ContextManager 
+        property realtime: Realtime    # Read-only
         agents: AgentsOperations
         beta: BetaOperations
         connections: ConnectionsOperations
@@ -41,9 +42,79 @@ namespace azure.ai.projects
             ) -> HttpResponse: ...
 
 
+    class azure.ai.projects.Realtime:
+
+        def __init__(self, client: AIProjectClient) -> None: ...
+
+        def connect(
+                self, 
+                *, 
+                agent_name: str, 
+                agent_session_id: Optional[str] = ..., 
+                agent_version_override: Optional[str] = ..., 
+                api_version: Optional[str] = ..., 
+                connection_url: Optional[str] = ..., 
+                credential_scopes: Optional[List[str]] = ..., 
+                extra_headers: Optional[Mapping[str, str]] = ..., 
+                extra_query: Optional[Mapping[str, str]] = ..., 
+                foundry_features: str = _VOICE_AGENT_FEATURE_HEADER, 
+                structured_inputs: Optional[str] = ..., 
+                **kwargs: Any
+            ) -> RealtimeConnectionManager: ...
+
+
+    class azure.ai.projects.RealtimeConnection: implements ContextManager 
+        property closed: bool    # Read-only
+
+        def __init__(self, connection: ClientConnection) -> None: ...
+
+        def __iter__(self) -> Iterator[ServerEvent]: ...
+
+        def __repr__(self) -> str: ...
+
+        def close(
+                self, 
+                *, 
+                code: int = 1000, 
+                reason: str = ""
+            ) -> None: ...
+
+        def recv(
+                self, 
+                *, 
+                timeout: Optional[float] = ...
+            ) -> ServerEvent: ...
+
+        def send(self, event: ClientEvent) -> None: ...
+
+
+    class azure.ai.projects.RealtimeConnectionManager: implements ContextManager 
+
+        def __init__(
+                self, 
+                *, 
+                agent_name: str, 
+                agent_session_id: Optional[str] = ..., 
+                agent_version_override: Optional[str] = ..., 
+                api_version: str, 
+                connection_url: Optional[str] = ..., 
+                credential: TokenCredential, 
+                credential_scopes: List[str], 
+                endpoint: str, 
+                extra_headers: Optional[Mapping[str, str]] = ..., 
+                extra_query: Optional[Mapping[str, str]] = ..., 
+                foundry_features: str, 
+                structured_inputs: Optional[str] = ..., 
+                **kwargs: Any
+            ) -> None: ...
+
+        def enter(self) -> RealtimeConnection: ...
+
+
 namespace azure.ai.projects.aio
 
     class azure.ai.projects.aio.AIProjectClient(AIProjectClientGenerated): implements AsyncContextManager 
+        property realtime: AsyncRealtime    # Read-only
         agents: AgentsOperations
         beta: BetaOperations
         connections: ConnectionsOperations
@@ -83,6 +154,75 @@ namespace azure.ai.projects.aio
             ) -> Awaitable[AsyncHttpResponse]: ...
 
 
+    class azure.ai.projects.aio.AsyncRealtime:
+
+        def __init__(self, client: AIProjectClient) -> None: ...
+
+        def connect(
+                self, 
+                *, 
+                agent_name: str, 
+                agent_session_id: Optional[str] = ..., 
+                agent_version_override: Optional[str] = ..., 
+                api_version: Optional[str] = ..., 
+                connection_url: Optional[str] = ..., 
+                credential_scopes: Optional[List[str]] = ..., 
+                extra_headers: Optional[Mapping[str, str]] = ..., 
+                extra_query: Optional[Mapping[str, str]] = ..., 
+                foundry_features: str = _VOICE_AGENT_FEATURE_HEADER, 
+                structured_inputs: Optional[str] = ..., 
+                **kwargs: Any
+            ) -> AsyncRealtimeConnectionManager: ...
+
+
+    class azure.ai.projects.aio.AsyncRealtimeConnection: implements AsyncContextManager 
+        property closed: bool    # Read-only
+
+        def __aiter__(self) -> AsyncIterator[ServerEvent]: ...
+
+        def __init__(
+                self, 
+                connection: ClientWebSocketResponse, 
+                session: ClientSession
+            ) -> None: ...
+
+        def __repr__(self) -> str: ...
+
+        async def close(
+                self, 
+                *, 
+                code: int = 1000, 
+                reason: str = ""
+            ) -> None: ...
+
+        async def recv(self) -> ServerEvent: ...
+
+        async def send(self, event: ClientEvent) -> None: ...
+
+
+    class azure.ai.projects.aio.AsyncRealtimeConnectionManager: implements AsyncContextManager 
+
+        def __init__(
+                self, 
+                *, 
+                agent_name: str, 
+                agent_session_id: Optional[str] = ..., 
+                agent_version_override: Optional[str] = ..., 
+                api_version: str, 
+                connection_url: Optional[str] = ..., 
+                credential: AsyncTokenCredential, 
+                credential_scopes: List[str], 
+                endpoint: str, 
+                extra_headers: Optional[Mapping[str, str]] = ..., 
+                extra_query: Optional[Mapping[str, str]] = ..., 
+                foundry_features: str, 
+                structured_inputs: Optional[str] = ..., 
+                **kwargs: Any
+            ) -> None: ...
+
+        async def enter(self) -> AsyncRealtimeConnection: ...
+
+
 namespace azure.ai.projects.aio.operations
 
     class azure.ai.projects.aio.operations.AgentEndpointConversationsOperations(GeneratedAgentEndpointConversationsOperations):

@@ -1,4 +1,4 @@
-apiMdSha256: 72dfd52cc4e653e857ac7c29948d0d1f3fb9c7d91dc78e9c2cd159458856c942
+apiMdSha256: 960131974f5eafb710e7cd9b31a8d4edfd220ac8709ed42dc129ad82e8ef412a
 packageVersion: 2.7.0b1
 parserVersion: 0.3.31
 pythonVersion: 3.13.2
@@ -22,6 +22,14 @@
 from .operations._patch import _OperationMethodHeaderProxy
 from .models._enums import _AgentDefinitionOptInKeys
 from .models._patch import _BETA_OPERATION_FEATURE_HEADERS, _FOUNDRY_FEATURES_HEADER_NAME, _has_header_case_insensitive
+from ._realtime import (
+    Realtime,
+    RealtimeConnection,
+    RealtimeConnectionManager,
+    ClientEvent,
+    ConversationItem,
+    ServerEvent,
+)
 
 _OPENAI_TRANSPORT_LOGGER_NAME = "azure.ai.projects.openai_transport"
 logger = logging.getLogger(__name__)
@@ -251,6 +259,7 @@ def __init__(
             )
 
         self.telemetry = TelemetryOperations(self)  # type: ignore
+        self._realtime: Optional[Realtime] = None
         # NOTE: voice-agent conversation reads (`agent_endpoint_conversations`) have round-tripped
         # between living directly on `self` (top-level) and being nested under `self.beta` across
         # several upstream TypeSpec regenerations. It is currently back to being a top-level,
@@ -260,6 +269,17 @@ def __init__(
         # `_BETA_OPERATION_FEATURE_HEADERS`/`BetaOperations.__init__` (which only applies to
         # `.beta`'s sub-clients).
 
+    @property
+    def realtime(self) -> Realtime:
+        """Realtime streaming entry point for voice agents.
+
+        :return: The realtime namespace, exposing ``connect(...)``.
+        :rtype: ~azure.ai.projects.Realtime
+        """
+        if self._realtime is None:
+            self._realtime = Realtime(self)
+        return self._realtime
+
     def _get_openai_api_key(self, kwargs: dict):
         """Resolve the API key for the OpenAI client.
 
@@ -523,6 +543,12 @@ def _log_request_body(self, request: httpx2.Request) -> None:
 
 __all__: List[str] = [
     "AIProjectClient",
+    "Realtime",
+    "RealtimeConnection",
+    "RealtimeConnectionManager",
+    "ClientEvent",
+    "ConversationItem",
+    "ServerEvent",
 ]  # Add all objects you want publicly available to users at this package level
 
 

@@ -34,6 +34,14 @@ from openai.types.eval_create_response import EvalCreateResponse
 from openai.types.shared_params.metadata import Metadata
 from ._client import AIProjectClient as AIProjectClientGenerated
 from .operations import TelemetryOperations
+from ._realtime import (
+    Realtime,
+    RealtimeConnection,
+    RealtimeConnectionManager,
+    ClientEvent,
+    ConversationItem,
+    ServerEvent,
+)
 from .models import (
     AzureAIBenchmarkPreviewEvalRunDataSource,
     AzureAIDataSourceConfig,
@@ -102,6 +110,8 @@ class OpenAI(OpenAIClient):
 
 class AIProjectClient(AIProjectClientGenerated):
     telemetry: TelemetryOperations
+    @property
+    def realtime(self) -> Realtime: ...
     def get_openai_client(
         self, agent_name: Optional[str] = None, **kwargs: Any  # pylint: disable=unused-argument
     ) -> OpenAI: ...
@@ -128,6 +138,12 @@ def _log_streaming_response_notice(logging_enabled: bool) -> bool: ...
 
 __all__: List[str] = [
     "AIProjectClient",
+    "Realtime",
+    "RealtimeConnection",
+    "RealtimeConnectionManager",
+    "ClientEvent",
+    "ConversationItem",
+    "ServerEvent",
 ]
 
 def patch_sdk() -> None: ...