Initial executor <> Plugins integration by EngHabu · Pull Request #6903 · flyteorg/flyte

EngHabu · 2026-02-08T17:06:15Z

Signed-off-by: Haytham Abuelfutuh haytham@afutuh.com

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Please add one or more of the following labels to categorize your PR:

added: For new features.
changed: For changes in existing functionality.
deprecated: For soon-to-be-removed features.
removed: For features being removed.
fixed: For any bug fixed.
security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

Screenshots

Check all the applicable boxes

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

Docs link

main
- Flyte 2 #6583
  - Fix vet issues #6901
    - Implement state service on top of etcd. #6902
      - Initial executor <> Plugins integration #6903 👈
        
        Switch to pflags #6904

machichima · 2026-02-23T09:11:24Z

+	// Resolve plugin from registry
+	p, err := r.PluginRegistry.ResolvePlugin(taskAction.Spec.TaskType)
+	if err != nil {


I'm thinking of moving this to a validation function, as well as validating other fields in taskAction, and put this validation function before we add the finalizer. In this case, if the validation failed, we can just not requeue it, and do not need to remove the finalizer.

We can do this in follow-up!

machichima · 2026-02-24T00:13:41Z

+		setCondition(ta, flyteorgv1.ConditionTypeProgressing, metav1.ConditionTrue,
+			flyteorgv1.ConditionReasonExecuting, info.Reason())


For future PR: we should probably add max retry and fail this if exceed

Signed-off-by: machichima <nary12321@gmail.com>

rely fully on task action conditions Signed-off-by: machichima <nary12321@gmail.com>

Signed-off-by: machichima <nary12321@gmail.com>

machichima · 2026-02-24T09:41:15Z

It was included in the.gitignore, I think some PR accidentally adding it back

flyte/.gitignore

Line 44 in b142044

**/rust/Cargo.lock

…/exec-plgns # Conflicts: # gen/rust/Cargo.lock

…nghabu/exec-plgns Signed-off-by: machichima <nary12321@gmail.com>

* refactor: add error log when fail to add finalizer Signed-off-by: machichima <nary12321@gmail.com> * fix: handle err when status update failed Signed-off-by: machichima <nary12321@gmail.com> * fix: remove phase is terminal check rely fully on task action conditions Signed-off-by: machichima <nary12321@gmail.com> * refactor: add event when failed to build context in finalize Signed-off-by: machichima <nary12321@gmail.com> * fix: reconcile again after update CR Signed-off-by: machichima <nary12321@gmail.com> * fix: only update status if changed Signed-off-by: machichima <nary12321@gmail.com> * build: remove tracked Cargo.lock Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: machichima <nary12321@gmail.com> Co-authored-by: machichima <nary12321@gmail.com>

* refactor: add error log when fail to add finalizer Signed-off-by: machichima <nary12321@gmail.com> * fix: handle err when status update failed Signed-off-by: machichima <nary12321@gmail.com> * fix: remove phase is terminal check rely fully on task action conditions Signed-off-by: machichima <nary12321@gmail.com> * refactor: add event when failed to build context in finalize Signed-off-by: machichima <nary12321@gmail.com> * fix: reconcile again after update CR Signed-off-by: machichima <nary12321@gmail.com> * fix: only update status if changed Signed-off-by: machichima <nary12321@gmail.com> * build: remove tracked Cargo.lock Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: machichima <nary12321@gmail.com> Co-authored-by: machichima <nary12321@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Implement state service on top of etcd. Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Initial executor <> Plugins integration Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Fix build and create_run.sh Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix: subscribe before listing Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * feat: store state to DB from state service watch Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix: skip update if state json does not change Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * feat: scope subscribe with parent action name Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * refactor: rename regularK8sClient to k8sClientWithoutWatch Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix: log when update channel full Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * refactor: remove unused function isChildOf Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * remove state service db persistant logic Signed-off-by: Alex Wu <c.alexwu@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix: a parent name link to multiple subscribers Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Initial executor <> Plugins integration (#6903) * refactor: add error log when fail to add finalizer Signed-off-by: machichima <nary12321@gmail.com> * fix: handle err when status update failed Signed-off-by: machichima <nary12321@gmail.com> * fix: remove phase is terminal check rely fully on task action conditions Signed-off-by: machichima <nary12321@gmail.com> * refactor: add event when failed to build context in finalize Signed-off-by: machichima <nary12321@gmail.com> * fix: reconcile again after update CR Signed-off-by: machichima <nary12321@gmail.com> * fix: only update status if changed Signed-off-by: machichima <nary12321@gmail.com> * build: remove tracked Cargo.lock Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: machichima <nary12321@gmail.com> Co-authored-by: machichima <nary12321@gmail.com> Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Cherry pick make local run work (#6943) * Switch to pflags Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> Signed-off-by: machichima <nary12321@gmail.com> * Let it rain Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> Signed-off-by: machichima <nary12321@gmail.com> * mostly working Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> Signed-off-by: machichima <nary12321@gmail.com> * Add literal-JSON converter and TranslatorService for RSJF-compliant schema translation. The converter handles bidirectional conversion Add literal-JSON converter and TranslatorService for RSJF-compliant schema translation Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> Signed-off-by: machichima <nary12321@gmail.com> * Add ShortName field to TaskActionSpec and populate it through the queue and watch pipelines. Pass inputUri and runOutputBase as parameters to CreateRun instead of computing them after DB insert. Fix WatchActions to subscribe before Add ShortName field to TaskActionSpec and populate it through the queue and watch pipelines Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> Signed-off-by: machichima <nary12321@gmail.com> * fix: Pass parent name to Subscribe() and add TODO Signed-off-by: machichima <nary12321@gmail.com> * fix: clean-up main and add StopWatching call Signed-off-by: machichima <nary12321@gmail.com> * build: update CR fields Signed-off-by: machichima <nary12321@gmail.com> * feat: build local env with k3d and run successfully Signed-off-by: machichima <nary12321@gmail.com> * docs: add TODO in CreateRun Signed-off-by: machichima <nary12321@gmail.com> * docs: add brief development guide Signed-off-by: machichima <nary12321@gmail.com> * build: make gen Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: machichima <nary12321@gmail.com> Co-authored-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Alex Wu <c.alexwu@gmail.com> Co-authored-by: Haytham Abuelfutuh <haytham@afutuh.com> Co-authored-by: Alex Wu <c.alexwu@gmail.com>

This was referenced Feb 8, 2026

Fix vet issues #6901

Merged

Implement state service on top of etcd. #6902

Closed

Flyte 2 #6583

Draft

Switch to pflags #6904

Closed

machichima reviewed Feb 23, 2026

View reviewed changes

machichima reviewed Feb 24, 2026

View reviewed changes

Comment thread executor/pkg/controller/taskaction_controller.go

machichima reviewed Feb 24, 2026

View reviewed changes

Comment thread executor/pkg/controller/taskaction_controller.go

popojk reviewed Feb 24, 2026

View reviewed changes

Comment thread executor/pkg/controller/taskaction_controller.go Outdated

machichima added 7 commits February 24, 2026 16:28

refactor: add error log when fail to add finalizer

42a904e

Signed-off-by: machichima <nary12321@gmail.com>

fix: handle err when status update failed

aa1c655

Signed-off-by: machichima <nary12321@gmail.com>

fix: remove phase is terminal check

3705ed5

rely fully on task action conditions Signed-off-by: machichima <nary12321@gmail.com>

refactor: add event when failed to build context in finalize

c7f20e5

Signed-off-by: machichima <nary12321@gmail.com>

fix: reconcile again after update CR

2c7d0bc

Signed-off-by: machichima <nary12321@gmail.com>

fix: only update status if changed

e3429fe

Signed-off-by: machichima <nary12321@gmail.com>

build: remove tracked Cargo.lock

bf144ad

Signed-off-by: machichima <nary12321@gmail.com>

machichima reviewed Feb 24, 2026

View reviewed changes

EngHabu and others added 3 commits February 24, 2026 09:57

Merge branch 'enghabu/state-etcd' into enghabu/exec-plgns

3d2d8d3

Merge remote-tracking branch 'origin/enghabu/exec-plgns' into enghabu…

f4d8dc6

…/exec-plgns # Conflicts: # gen/rust/Cargo.lock

Merge branch 'enghabu/state-etcd' of github.com:flyteorg/flyte into e…

c524570

…nghabu/exec-plgns Signed-off-by: machichima <nary12321@gmail.com>

machichima approved these changes Feb 25, 2026

View reviewed changes

machichima marked this pull request as ready for review February 25, 2026 09:14

machichima mentioned this pull request Feb 25, 2026

[V2] Move validation to a validation function #6934

Closed

machichima merged commit 9615fed into enghabu/state-etcd Feb 26, 2026
5 checks passed

machichima deleted the enghabu/exec-plgns branch February 26, 2026 03:13

AdilFayyaz mentioned this pull request Mar 10, 2026

Add TaskAction validation before finalizer registration #7006

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial executor <> Plugins integration#6903

Initial executor <> Plugins integration#6903
machichima merged 10 commits intoenghabu/state-etcdfrom
enghabu/exec-plgns

EngHabu commented Feb 8, 2026 •

edited by github-actions Bot

Loading

Uh oh!

Uh oh!

Uh oh!

machichima Feb 23, 2026

Uh oh!

Uh oh!

machichima Feb 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

machichima Feb 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		setCondition(ta, flyteorgv1.ConditionTypeProgressing, metav1.ConditionTrue,
		flyteorgv1.ConditionReasonExecuting, info.Reason())

Conversation

EngHabu commented Feb 8, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Setup process

Screenshots

Check all the applicable boxes

Related PRs

Docs link

Uh oh!

Uh oh!

Uh oh!

machichima Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

machichima Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

machichima Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

EngHabu commented Feb 8, 2026 •

edited by github-actions Bot

Loading