Skip to content

Conversation

@lstein
Copy link
Collaborator

@lstein lstein commented Dec 24, 2025

Summary

Due to a core issue involving torch memory mapping of GGUF, attempts to install GGUF models on Windows platforms fail with an error that rmtree() cannot remove the temporary directory created to hold the downloaded model. This PR fixes the issue in several ways:

  1. It replaces torch.from_numpy(tensor.data) in the GGUF reader with torch.from_numpy(tensor.data.copy()). This makes a copy of the torch data so that the memory-mapped file is not opened. This is the core issue and should fix the problem. But just in case...
  2. It wraps rmtree() and move() calls in loops that call garbage collection and insert short delays over multiple tries.

Related Issues / Discussions

See Discord discussion starting around message https://discord.com/channels/1020123559063990373/1049495067846524939/1453426867733401663.

QA Instructions

On a Windows system:

  1. Pull this PR and restart the server.
  2. Go to the Model Manager and install the GGUF at https://huggingface.co/wikeeyang/SRPO-Refine-Quantized-v1.0/resolve/main/Flux1-Dev-SRPO-v1-Q4_1.gguf?download=true (or any other GGUF model supported by InvokeAI)
  3. The model should install and run properly.
  4. Examine the log file. You should see a warning like this: Failed to remove temporary directory XXXXXX: Permission Denied.... It will be removed on next server start
  5. Restart the server. You should see a warning like Removing dangling temporary directory XXXXXX

Merge Plan

Small change. Should be a simple merge.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@github-actions github-actions bot added python PRs that change python files services PRs that change app services labels Dec 24, 2025
@github-actions github-actions bot added the backend PRs that change backend files label Dec 26, 2025
@lstein lstein enabled auto-merge (squash) December 26, 2025 01:58
@lstein lstein merged commit b9493dd into main Dec 26, 2025
13 checks passed
@lstein lstein deleted the lstein/bugfix/gguf-rmtree-on-windows branch December 26, 2025 02:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend PRs that change backend files python PRs that change python files services PRs that change app services

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants