Skip to content

Conversation

@blessedcoolant
Copy link
Collaborator

Summary

Implement the node and UI to generate PBR Maps from a single given image. Based on: https://github.com/joeyballentine/Material-Map-Generator

355001073-2e8ad8ad-3976-4225-84d2-eb50f5911fb2

All images now have a new context menu item called "Filters" where we can add individual image filters and image actions starting with PBR Maps.

opera_sQUZBcCqLj

QA Instructions

  1. Right click any image. Try with various images just to make sure nothing breaks.
  2. Filters > Create PBR Maps
  3. The models should auto download the first time.
  4. Three new images should get created in your board - normal map, roughness map and a displacement map.

Merge Plan

Test and merge. I'd just note that we might want to host these models somewhere ourselves incase to make sure it does not break in the future.

@github-actions github-actions bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files frontend PRs that change frontend files labels Dec 25, 2025
@blessedcoolant blessedcoolant changed the title Implement PBR Nodes Implement PBR Maps Node Dec 25, 2025
@lstein
Copy link
Collaborator

lstein commented Dec 28, 2025

I gave the PBR PR a try just now. The filter option appeared, but when I tried to generate the PBR maps, the backend gave me the following error:

[2025-12-27 21:12:09,787]::[InvokeAI]::ERROR --> Error while invoking session c71ec4e8-f3a2-4ca4-b48a-f81ffc113ffd, invocation d0184b5a-0082-4868-bd15-1073f20b3635 (pbr_maps): Weights only load failed. In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
Please file an issue with the following so that we can make `weights_only=True` compatible with your use case: WeightsUnpickler error: Unsupported operand 10

Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.
[2025-12-27 21:12:09,787]::[InvokeAI]::ERROR --> Traceback (most recent call last):
  File "/home/lstein/Projects/InvokeAI/invokeai/app/services/session_processor/session_processor_default.py", line 130, in run_node
    output = invocation.invoke_internal(context=context, services=self._services)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lstein/Projects/InvokeAI/invokeai/app/invocations/baseinvocation.py", line 244, in invoke_internal
    output = self.invoke(context)
             ^^^^^^^^^^^^^^^^^^^^
  File "/home/lstein/Projects/InvokeAI/invokeai/app/invocations/pbr_maps.py", line 36, in invoke
    context.models.load_remote_model(NORMAL_MAP_MODEL, loader) as normal_map_model,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lstein/Projects/InvokeAI/invokeai/app/services/shared/invocation_context.py", line 543, in load_remote_model
    return self._services.model_manager.load.load_model_from_path(model_path=model_path, loader=loader)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lstein/Projects/InvokeAI/invokeai/app/services/model_load/model_load_default.py", line 126, in load_model_from_path
    raw_model = loader(model_path)
                ^^^^^^^^^^^^^^^^^^
  File "/home/lstein/Projects/InvokeAI/invokeai/app/invocations/pbr_maps.py", line 33, in loader
    return PBRMapsGenerator.load_model(model_path, TorchDevice.choose_torch_device())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lstein/Projects/InvokeAI/invokeai/backend/image_util/pbr_maps/pbr_maps.py", line 28, in load_model
    state_dict = torch.load(model_path.as_posix(), map_location="cpu")
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lstein/invokeai-main/.venv/lib/python3.12/site-packages/torch/serialization.py", line 1548, in load
    raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
_pickle.UnpicklingError: Weights only load failed. In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
Please file an issue with the following so that we can make `weights_only=True` compatible with your use case: WeightsUnpickler error: Unsupported operand 10

Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.

I've got torch 2.7.1+cu128 installed, which I need to support the 5060 ti graphics card.

@blessedcoolant
Copy link
Collaborator Author

@lstein I added safe globals instead of just setting weights_only=False .. Let me know if this works. If not, I can set the weights only to be False as it was in the previous versions.

@lstein
Copy link
Collaborator

lstein commented Dec 28, 2025

@copilot, I'm getting this error:

  File "/home/lstein/invokeai-main/.venv/lib/python3.12/site-packages/torch/serialization.py", line 1548, in load
    raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
_pickle.UnpicklingError: Weights only load failed. In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
Please file an issue with the following so that we can make `weights_only=True` compatible with your use case: WeightsUnpickler error: Unsupported operand 10

Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements a PBR (Physically Based Rendering) Maps generator that creates Normal, Roughness, and Displacement maps from a single input image. The implementation is based on the Material-Map-Generator project and adds a new "Filters" context menu to the image gallery for accessing image processing actions.

Key changes:

  • Backend PBR maps generation using neural network models (RRDB architecture)
  • New node invocation pbr_maps for workflow integration
  • Frontend UI with context menu integration and graph building
  • Model loading infrastructure for remote model downloads

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
invokeai/app/invocations/pbr_maps.py Implements the PBR Maps invocation node with image processing logic
invokeai/backend/image_util/pbr_maps/pbr_maps.py Core PBR maps generator with model loading and inference
invokeai/backend/image_util/pbr_maps/architecture/pbr_rrdb_net.py Neural network architecture definition for PBR models
invokeai/backend/image_util/pbr_maps/architecture/block.py Reusable neural network building blocks and layers
invokeai/backend/image_util/pbr_maps/utils/image_ops.py Image processing utilities for tiling and seamless operations
invokeai/frontend/web/src/features/nodes/util/graph/filters/buildPBRFilterGraph.ts Graph builder for PBR filter workflow
invokeai/frontend/web/src/features/gallery/components/ContextMenu/MenuItems/ContextMenuItemFiltersSubMenu.tsx New Filters submenu component for image context menu
invokeai/frontend/web/src/features/gallery/components/ContextMenu/SingleSelectionMenuItems.tsx Integration of Filters submenu into context menu
invokeai/frontend/web/src/app/store/middleware/listenerMiddleware/listeners/addPBRFilterListener.ts Event listener for PBR processing requests with queue management
invokeai/frontend/web/src/app/store/store.ts Registration of PBR filter listener
invokeai/frontend/web/public/locales/en.json Translation strings for PBR maps feature
invokeai/frontend/web/src/services/api/schema.ts Auto-generated API schema updates

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@blessedcoolant
Copy link
Collaborator Author

Resolved all Copilot suggestions and ported over to using our self-hosted safetensors models instead. Working well on my end. Let me know if the issue with the 5000 series models is resolved with the non pickle files. All models auto download. 20 MB each -> 40 overall.

Copy link
Collaborator

@lstein lstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is working as advertised

@lstein lstein enabled auto-merge (squash) December 29, 2025 01:57
@lstein lstein merged commit 1675712 into invoke-ai:main Dec 29, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend PRs that change backend files frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants