Skip to content

sfs: resource pool create times out after a fixed 10 minutes, and the tainted retry collides with the existing pool #1737

Description

@devpie

Summary

stackit_sfs_resource_pool cannot create a pool that STACKIT needs more than 10 minutes to
provision. The resource exposes no timeouts block, so the wait handler's default of 10
minutes cannot be raised from configuration. When the wait expires, Terraform marks the
object tainted and replaces it on the next run, which collides with the pool that STACKIT
already created.

Versions

Provider stackitcloud/stackit v0.113.0
SDK github.com/stackitcloud/stackit-sdk-go/services/sfs v0.11.0 (unchanged in v0.11.1)
OpenTofu 1.12.6, darwin_arm64
Region eu01, availability zone eu01-m

Where the 10 minutes come from

CreateResourcePoolWaitHandler sets a fixed timeout
(sfs/v1api/wait/wait.go, v0.11.0):

handler.SetTimeout(10 * time.Minute)

The provider calls it without overriding that value
(sfs/resourcepool/resource.go, v0.113.0):

response, err := wait.CreateResourcePoolWaitHandler(ctx, r.client.DefaultAPI, projectId, region, *resourcePool.ResourcePool.Id).
	WaitWithContext(ctx)

docs/resources/sfs_resource_pool.md documents no timeouts attribute, so there is no way
to raise it from configuration. Update and delete have the same fixed 10 minutes.

What happens when it expires

Immediately before waiting, the provider writes the identifiers to state on purpose
(same file, L288–L293):

// Write id attributes to state before polling via the wait handler - just in case anything goes wrong during the wait handler
ctx = utils.SetAndLogStateFields(ctx, &resp.Diagnostics, &resp.State, map[string]any{
	"project_id":       projectId,
	"region":           region,
	"resource_pool_id": *resourcePool.ResourcePool.Id,
})

Those identifiers do survive the failed apply. fwserver.CreateResource assigns
resp.NewState = &createResp.State before it inspects the diagnostics
(terraform-plugin-framework v1.19.0, server_createresource.go#L131),
so the partial entry is written to state.

Terraform then marks that object tainted, because the create failed partway
(Terraform docs, Tainted Objects):
"Terraform replaces objects in a tainted state during the next plan or apply operation."
Replacing means destroy plus create, not a refresh of the existing pool — so the recreate
collides with the pool STACKIT already has.

What I observed in my run:

terraform.tfstate.backup  serial 5  ->  stackit_sfs_resource_pool.this present, 3 attributes set
terraform.tfstate         serial 6  ->  entry gone
Error: Error creating resource pool
  with stackit_sfs_resource_pool.this,
  on main.tf line 191, in resource "stackit_sfs_resource_pool" "this":
Calling API: 409 Conflict, status code 409, Body: {"error_description":"An error occurred",
"type":"storage.stackit.cloud/conflict","detail":"resource pool 'sfs-snap-probe' already exists",
"status_code":409}

Trace ID: "fe474fe6eb091d232dfef6cbfeeb9c05"

I did not keep the logs of the destroy step, so I cannot say which step removed the entry
from state. Recovering needs untaint plus a re-apply, or a manual import with an id the
operator has to dig out of the portal or a previous state file. The error message names
neither.

Reproduce

resource "stackit_sfs_resource_pool" "this" {
  project_id        = var.project_id
  name              = "sfs-snap-probe"
  availability_zone = "eu01-m"
  performance_class = "Standard"
  size_gigabytes    = 1024
  ip_acl            = ["10.60.10.0/25"]

  snapshots_are_visible = true
}

Apply against a project attached to an SNA with routing tables enabled. In my runs the
create is still polling well past a minute and the previous run gave up at the 10 minute
mark. Whether a smaller pool stays under the limit I have not tested — the point is that
the limit is not reachable from configuration either way.

Suggested fix

  1. Expose a timeouts block on stackit_sfs_resource_pool (create/read/update/delete) and
    bound the wait through the context, as 19 other files in this provider already do — for
    example stackit/internal/services/dremio/instance/resource.go, which reads the waiter
    default via GetTimeout(), adds core.DefaultTimeoutMargin and calls
    context.WithTimeout.
    handler.SetTimeout alone does not work here: its own doc comment says "This only has an
    effect, if there's no timeout/deadline set on the context", and WaitWithContext applies
    it only under if _, ok := ctx.Deadline(); !ok
    (core v0.26.0, wait/wait.go).

  2. Name the recovery in the wait-timeout error message. The pool exists, its identifiers are
    in state, and the operator needs the import id [project_id],[region],[resource_pool_id]
    or an untaint to get a refresh instead of a replacement. Today the message is only
    resource pool creation waiting: %v.

An earlier version of this issue suggested a third point, "leave the resource in state
rather than dropping it". I withdraw it: the provider already does that, and a provider
cannot clear the taint that Terraform sets. It also suggested documenting the import id
format, which docs/resources/sfs_resource_pool.md already does.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions