[azure-ai-projects] Add Agent Insights Samples - #48889
Open
seangayler-msft wants to merge 33 commits into
Open
Conversation
|
Azure Pipelines: 11 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
seangayler-msft
marked this pull request as ready for review
September 3, 2026 20:00
seangayler-msft
force-pushed
the
sean/agent-insights-samples
branch
from
September 3, 2026 20:02
a6a5773 to
2c62b9c
Compare
seangayler-msft
marked this pull request as draft
September 3, 2026 20:03
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The reviewed changes have no unresolved issues blocking approval.
Pull request overview
Adds a synchronous Agent Insights sample demonstrating a safe monitor lifecycle.
Changes:
- Demonstrates monitor creation, retrieval, listing, and cleanup.
- Registers sample testing and LLM validation.
- Documents the sample in the changelog.
File summaries
| File | Description |
|---|---|
sdk/ai/azure-ai-projects/tests/samples/test_samples.py |
Registers the sample, pending an HTTP recording. |
sdk/ai/azure-ai-projects/tests/samples/llm_instructions.py |
Adds Agent Insights output validation mapping. |
sdk/ai/azure-ai-projects/samples/agent_insights/sample_agent_insights_basic.py |
Implements the monitor lifecycle sample. |
sdk/ai/azure-ai-projects/CHANGELOG.md |
Announces the new sample. |
Review details
- Files reviewed: 4/4 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
seangayler-msft
marked this pull request as ready for review
September 3, 2026 20:44
|
Azure Pipelines: Successfully started running 1 pipeline(s). 10 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
seangayler-msft
force-pushed
the
sean/agent-insights-samples
branch
from
September 4, 2026 17:18
2c62b9c to
36680dd
Compare
seangayler-msft
marked this pull request as draft
September 4, 2026 17:18
seangayler-msft
changed the title
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Live-test authentication and dependency ordering are broken, and fixture code is incompatible with supported Python 3.10.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 15/15 changed files
- Comments generated: 4
- Review effort level: Balanced
Member
Author
|
/azp run |
|
Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command. |
Member
Author
|
/azp run |
|
Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command. |
seangayler-msft
marked this pull request as ready for review
September 4, 2026 18:05
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The scheduled sample can report a missing next-run timestamp despite its documented and tested contract.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
sdk/ai/azure-ai-projects/samples/agent_insights/sample_agent_insights_scheduled.py:89
- This can print
Next scheduled run: None, so the sample does not actually read the next run time it promises to demonstrate. The new unit test explicitly acceptsnext_run=None(test_agent_insights_samples.py:466-490), while the recorded-sample validator rejects that same output (sample_test_helpers.py:44-50). Treat a missing timestamp as an error or poll/get the enabled monitor until a timestamp is available, and update the unit test to require that behavior.
- Files reviewed: 23/23 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Require agents without existing monitors. Delete on-demand monitors only after success; retain bounded scheduled teardown with ten-second polling. Update cleanup tests and maintainer guidance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The polling and Azure resource changes depend on live service behavior that cannot be fully validated from the available recordings.
Review details
- Files reviewed: 23/23 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
It combines public polling behavior changes with live Azure resource provisioning and recorded integration workflows, warranting final human validation.
Review details
- Files reviewed: 24/24 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Use default final-response handling, preserve feature headers throughout polling, and record the simplified sample workflows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc
Remove duplicate error matrices and redundant mocked coverage while retaining polling headers, cancellation, cleanup races, future severity, and recording safeguards. Reuse the existing playback sleep fixture. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc
Register temporary external agents, seed fictional telemetry, and clean up owned resources in each sample. Remove the separate provisioning fixture and keep focused lifecycle coverage. Recording refresh remains deferred. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The referenced recordings do not cover the new lifecycle, and recording sanitization would prevent deterministic playback.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 19/19 changed files
- Comments generated: 3
- Review effort level: Balanced
| # Sanitize checkpoint IDs in URLs and response bodies | ||
| add_general_regex_sanitizer(regex=r"ftchkpt-[a-f0-9]+", value="sanitized-checkpoint-id") | ||
|
|
||
| add_agent_insights_sanitizers() |
| "AssetsRepoPrefixPath": "python", | ||
| "TagPrefix": "python/ai/azure-ai-projects", | ||
| "Tag": "python/ai/azure-ai-projects_81db4bd08f" | ||
| "Tag": "python/ai/azure-ai-projects_ca13a42c98" |
Record the complete temporary-agent sample lifecycle, scope telemetry redactions, and preserve trace relationships during playback. Align the package, changelog, sample requirements, and generated API metadata with version 2.7.0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Samples
Adds two self-contained samples under
samples/agent_insights/. Each registers a temporary external agent, emits fictional traces, waits for ingestion, and deletes its own monitor and agent infinally.Setup and cleanup are simple functions at the bottom of each sample. The project, connected Application Insights, analysis-model deployment, and permissions must already exist. The samples do not deploy infrastructure or change roles.
Supporting changes
tests/agent_insights/.Foundry-Featureson polling requests and the final GET.Cancel VS Cancelled Patch Explanation
Cancellation is a core Agent Insights capability. The service returns
cancelled, but Azure Core's default polling logic recognizescanceledinstead. Without the adapter, polling can continue after a run is cancelled. The patch makes cancellation terminal for the polling engine while keeping the publicpoller.status()value ascancelled.