Skip to content

feat(experimentation): experiment rollout with multivariate segment override#7851

Open
gagantrivedi wants to merge 1 commit into
mainfrom
feat/experiment-rollout
Open

feat(experimentation): experiment rollout with multivariate segment override#7851
gagantrivedi wants to merge 1 commit into
mainfrom
feat/experiment-rollout

Conversation

@gagantrivedi

@gagantrivedi gagantrivedi commented Jun 23, 2026

Copy link
Copy Markdown
Member

Thanks for submitting a PR! Please check the boxes below:

  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Adds an experiment rollout to experiment creation. Posting an experiment_rollout block alongside a new experiment provisions, in one atomic create:

  • a PERCENTAGE_SPLIT system segment (is_system_segment=True, so it's hidden from the segments API/UI and audit log) gating the experiment cohort by rollout_percentage;
  • a segment override on the experiment's multivariate feature carrying the control value and the per-variant multivariate_feature_state_values split.

Supporting change in the versioning layer: FlagChangeSet / SegmentOverrideChangeSet and update_flag / update_flag_v2 now carry optional multivariate allocations (upsert-style), so segment-override writes set the variant split correctly for both v1 and v2 feature-versioning environments. This also benefits the existing update-flag-v2 endpoint.

Not yet included (intentional)

  • experiment_rollout is write-only — no read representation for the UI yet.
  • No teardown of the rollout segment/override when an experiment completes.

How did you test this code?

Unit tests (all green locally), plus mypy, ruff, and the test linter:

  • Versioning service: update_flag / update_flag_v2 set/upsert/remove multivariate allocations on segment overrides, for v1 and v2 environments.
  • Rollout services: create_experiment_rollout builds the system %-split segment + override; update_experiment_rollout updates it and is blocked while running/completed and when no rollout exists.
  • API: creating an experiment with experiment_rollout (incl. invalid MV option → 400), the rollout PATCH action (happy path, running → 400, invalid MV → 400), and rejecting a rollout sent to the experiment update endpoint.

@vercel

vercel Bot commented Jun 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Jun 23, 2026 12:03pm
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
flagsmith-frontend-preview Ignored Ignored Preview Jun 23, 2026 12:03pm
flagsmith-frontend-staging Ignored Ignored Preview Jun 23, 2026 12:03pm

Request Review

@github-actions github-actions Bot added api Issue related to the REST API feature New feature or request labels Jun 23, 2026
Comment thread api/experimentation/views.py Fixed
@codecov

codecov Bot commented Jun 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.60%. Comparing base (abfdc80) to head (7c3d4fa).
⚠️ Report is 12 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7851      +/-   ##
==========================================
+ Coverage   98.44%   98.60%   +0.15%     
==========================================
  Files        1467     1473       +6     
  Lines       57327    57669     +342     
==========================================
+ Hits        56436    56863     +427     
+ Misses        891      806      -85     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gagantrivedi

Copy link
Copy Markdown
Member Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for experiment rollouts by adding a rollout_segment to the Experiment model, implementing rollout creation and update services, and extending the versioning service to handle multivariate value updates. The review feedback highlights several critical issues: a truthiness check in the serializer prevents clearing multivariate values with an empty list; the rollout update service lacks a database transaction, risking data inconsistency; and the multivariate update loop is susceptible to duplicate creation or integrity errors if duplicate option IDs are provided.

Comment thread api/features/feature_states/serializers.py
Comment thread api/experimentation/services.py
Comment thread api/experimentation/services.py
Comment thread api/features/versioning/versioning_service.py Outdated
@github-actions github-actions Bot added feature New feature or request docs Documentation updates and removed feature New feature or request docs Documentation updates labels Jun 23, 2026
@gagantrivedi gagantrivedi force-pushed the feat/experiment-rollout branch from 461a758 to f90171e Compare June 23, 2026 07:56
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Jun 23, 2026
@gagantrivedi gagantrivedi force-pushed the feat/experiment-rollout branch from f90171e to 851a93b Compare June 23, 2026 10:15
@github-actions github-actions Bot added the docs Documentation updates label Jun 23, 2026
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Jun 23, 2026
@gagantrivedi gagantrivedi force-pushed the feat/experiment-rollout branch from 851a93b to b72bb67 Compare June 23, 2026 10:32
@github-actions github-actions Bot added the docs Documentation updates label Jun 23, 2026
@github-actions github-actions Bot removed feature New feature or request docs Documentation updates labels Jun 23, 2026
@github-actions github-actions Bot added feature New feature or request and removed docs Documentation updates labels Jun 23, 2026
@github-actions github-actions Bot added docs Documentation updates feature New feature or request and removed feature New feature or request docs Documentation updates labels Jun 23, 2026
…nt override

Add an experiment_rollout block to experiment creation that provisions a
percentage-split system segment plus a segment override (with multivariate
allocations) on the experiment's feature, and a PATCH rollout action to update
it. Extends the versioning service so single and bulk flag updates can carry
multivariate values for v1 and v2 environments.
@gagantrivedi gagantrivedi force-pushed the feat/experiment-rollout branch from 0202454 to 7c3d4fa Compare June 23, 2026 12:03
@github-actions github-actions Bot added docs Documentation updates feature New feature or request and removed feature New feature or request docs Documentation updates labels Jun 23, 2026
@gagantrivedi gagantrivedi marked this pull request as ready for review June 23, 2026 12:08
@gagantrivedi gagantrivedi requested review from a team as code owners June 23, 2026 12:08
@gagantrivedi gagantrivedi requested review from emyller and removed request for a team June 23, 2026 12:08
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request labels Jun 23, 2026
@github-actions

github-actions Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-e2e:pr-7851 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api-test:pr-7851 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-frontend:pr-7851 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-api:pr-7851 Finished ✅ Results
ghcr.io/flagsmith/flagsmith:pr-7851 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-private-cloud:pr-7851 Finished ✅ Results

@github-actions

github-actions Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  33 seconds
commit  7c3d4fa
info  🔄 Run: #17743 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  37.9 seconds
commit  7c3d4fa
info  🔄 Run: #17743 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  4 passed

Details

stats  4 tests across 4 suites
duration  3.5 seconds
commit  7c3d4fa
info  🔄 Run: #17743 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  59.4 seconds
commit  7c3d4fa
info  🔄 Run: #17743 (attempt 1)

@github-actions

Copy link
Copy Markdown
Contributor

Visual Regression

19 screenshots compared. See report for details.
View full report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Issue related to the REST API feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants