Skip to content

feat(ktx): add initial ktx input/output support.#5185

Open
walcht wants to merge 44 commits into
AcademySoftwareFoundation:mainfrom
walcht:main
Open

feat(ktx): add initial ktx input/output support.#5185
walcht wants to merge 44 commits into
AcademySoftwareFoundation:mainfrom
walcht:main

Conversation

@walcht

@walcht walcht commented May 5, 2026

Copy link
Copy Markdown

Description

Fixes #2329. Add initial KTX2 format support (closest similar plugin/format in OIIO is DDS).

This KTX plugin support obviously nullifies the benefits of using KTX in the
first place. That being said, this plugin is still useful so that end users
don't have to convert back and forth between KTX <-> supported format (e.g., PNG).
It is also useful to convert to and from KTX2 format using OIIO.

An example usecase would be Blender and its glTf import/export plugin.
(see KhronosGroup/glTF-Blender-IO#1896).

Ideally, at some point in the future, OIIO may introduce a new API to accomodate
texture formats that are mainly used for fast texture uploads to GPUs.

For details on KTX2 and what this PR supports and what it doesn't (alongside a
description of current limitations), see src/ktx.imageio/README.md (copied parts
of it at the end of this).

Tests:

Added initial testsuite but only for the input (i.e., info_command) part of the plugin.
Ideally, after fixing CI issues, the tests will be adjusted for ktxouput.

Checklist:

  • I have read the guidelines on contributions and code review procedures.
  • I have read the Policy on AI Coding Assistants
    => Haven't used any AI coding assistant tools in any capacity whatsoever.
  • I have updated the documentation if my PR adds features or changes
    behavior.
  • I am sure that this PR's changes are tested in the testsuite.
  • I have run and passed the testsuite in CI before submitting the
    PR, by pushing the changes to my fork and seeing that the automated CI
    passed there.
    => The 3 failing MacOS CIs are not caused by this PR (Jxl and OpenCV-related).
  • My code follows the prevailing code style of this project and I
    fixed any problems reported by the clang-format CI test.
  • If I added or modified a public C++ API call, I have also amended the
    corresponding Python bindings. If altering ImageBufAlgo functions, I also
    exposed the new functionality as oiiotool options.
    => No new API is introduced. This is just a plugin.

The sections below are copied from src/ktx.imageio/README.md. These will be updated
as this PR progresses.

Supported Encoders/Decoders

  • Supported/Tested texture kinds:

    • SINGLE_TEXTURE_1D (TODO)
    • SINGLE_TEXTURE_2D
    • SINGLE_TEXTURE_3D
    • CUBEMAP_TEXTURE (TODO)
    • ARRAY_TEXTURE_1D
    • ARRAY_TEXTURE_2D
    • [ ] ARRAY_TEXTURE_3D (not planned)
    • [ ] ARRAY_TEXTURE_CUBEMAP (not planned)
  • Supported/Tested raw VkFormats (decoder + encoder):

    • VK_FORMAT_R8_UNORM
    • VK_FORMAT_R8G8_SRGB
    • VK_FORMAT_R8G8B8_SRGB
    • VK_FORMAT_R8G8B8A8_SRGB
  • Block-compressed formats (decoder + encoder):

  • Basis Universal schemes (encoder + decoder):

    • UASTC
    • ETC1S
  • Supercompression schemes (decompressor + compressor):

    • ZLIB
    • ZSTD

Dependencies (only one - libktx)

  • libktx: for general KTX@ format support (loading of KTX2 files, transcoding
    support, supercompression decompression support, etc.).

    • Commit hash: see OpenImageIO/src/cmake/build_Ktx.cmake
    • License: Many subresources. All permissive except lib/etcdec.cxx's license (ETC is not used in PR).

This PR should only be merged after the release of libktx 5.0.0.

Resources

Add limited support for input and output for the KTX2
format.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@walcht

walcht commented May 6, 2026

Copy link
Copy Markdown
Author

Update: I'm working on integrating BCn decoders/encoders directly within libktx so that this PR gets significantly simpler (e.g., from 12_000 lines most of which are from dependencies to circa 2_000).

walcht and others added 2 commits June 22, 2026 13:27
*Significantly simplify KTX support by relying on libktx to
decode/encode GPU-compressed formats such as BCn, ASTC, and ETC.
Consequently, BCn and ETC dependencies are removed. All future required
encoders/decoders should be implemented in libktx and not here to keep
this is as simple, maintainable, and short as possible.

*Remove ETC sub-dependency from local libktx build.

*Update ktx README.md documentation to reflect that only libktx
dependency is and should be used (i.e., no additional dependencies
should be added).

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@walcht

walcht commented Jun 22, 2026

Copy link
Copy Markdown
Author

Update2: significantly simplified the whole PR. Now only dependency is libktx (since I offloaded most work to libktx via a PR that will be probably merged soon).

Can you please not approve workflows unless I explicitly request a workflow run? (to avoid filling the actions log with CIs that I know will certainly fail).

walcht added 2 commits June 22, 2026 15:20
*IntelLLVM is simply not supported by libktx.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@lgritz

lgritz commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Can you please not approve workflows unless I explicitly request a workflow run? (to avoid filling the actions log with CIs that I know will certainly fail).

Sure, in this case I can do so, but only because you are a a first-time contributor to this project, so admins have to approve workflow runs on the project's main account. Once you have any PR accepted and merged, the workflows will run automatically.

Here is the general solution that you or others can do, in this situation and after you no longer need admin approval:

Let's say you have pushed to your fork's branch "mypr" and submitted a PR to the project from that branch. (Aside: I strongly recommend working in topic branches, and never submitting a PR from your "main".)

After submitting the PR, you realize you need to iterate (you want to add things, or CI is not passing and you need to make repairs, or you have review comments to address), but you don't want to do a series of many many pushes as you go, each of which will trigger a CI run on the main branch as well as sending email alerts to everybody "watching" this project, since pushing to your branch is technically a PR update.

Instead, push to a different branch!

git push origin mypr:test

This pushes your fork's "mypr" branch to your fork in a "test" branch. This will run the CI on your fork in your account (allowing you to see if CI passes for the changes you've made), but since your "test" branch is not associated with the PR, it will not send the rest of us alerts and will not trigger a CI run on the project's main repo account. Then when you finally have it right and are ready for the rest of us to see it, you can git push origin mypr to update "mypr", which the system will know is the branch you are making the PR from.

*Add to_native* conversion to KTX2 output similar to how JPEG and PNG OIIO
outputs are written.

*Apply clang-format-17 as opposed to the previously applied clang-format
v18 which caused the CI to fail.

*Update CMAKE_VERSION in CIs to minimum version required by libktx
(i.e., 3.23).

*Fix uninitialized structs passed to libktx.

*Other misc clean-ups and minor refactoring.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@walcht

walcht commented Jun 29, 2026

Copy link
Copy Markdown
Author

Question about older CMake version that is failing some CIs and icpx:

  1. Some CIs require a relatively old version of CMake (3.18.3) but libktx (the library through which KTX2 support is achieved) requires CMake >= 3.23. (e.g., CI oldest clang11/C++17 py3.9 exr3.1 ocio2.3 / c++17 py3.9).
    In case I may not be able to lower libktx's minimum CMake version (e.g., request refused or simply not possible/requires too many changes) => I couldn't lower CMake version simply because libktx dependencies also require CMake >= 3.20. How should I proceed with this? => Solved. Used older libktx version (4.3.2) for these CIs.

  2. ICPX compiler is not supported by libktx (haven't tried compiling it with it yet but it is definitely not supported as per the main CMakeList.txt of libktx). How should I proceed here? => Solved. I have just followed libuhrd and overridden C/C++ compilers for libktx in ICX CI.

These are the only* remaining CI issues.

*: there is another one that is Intel-MacOS only but there is a PR already to fix it in libktx.

All CI issues are now solved. Will push updates here when sufficient testing is done.

walcht added 19 commits July 4, 2026 17:46
…actoring

*KTX does not support compilation with Intel's C++ compiler ICX hence
why an override option is provided and is used in ICX CI.

*Do all decompression/decoding/deflation in open() rather than in
seek_subimage()

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*Due to missing 'alpha' support (which obviously exists in libktx), unit
testing was failing (due to unexpected number of channels).

*Misc cleanup; removal of ktxTexture* member field and only using
ktxTexture2*.

*Remove MSVC/GNU/Clang checks on libktx dependency in externalpackages.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*Previously, a buffer allocated via a malloc by libktx was freed using
RAII via std::unique_ptr's delete[] which doesn't match the allocator.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*libktx > 4.3.2 requires CMake >= 3.22. The code changes required for
this will also be applied accordingly (e.g., older libktx versions don't
support direct BCn encoding/decoding).

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*Previously, libktx version was hardcoded to v5.0.0 now it is based on
the CMake-set variable Ktx_VERSION.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*malloc has to be matched by a call to free not a call to delete.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*Attempting to address old CMake version with 'oldest' CIs.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*Point to not-yet-merged PR in libktx only for Intel-based MacOS runner.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
This is added so that CIs on older runners/compilers/cmake can pass.
Newer version of libktx require CMake >= 3.22 which these runners cannot
install.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
ktxBasisParams is vastly different starting from libktx >= 5.0.0.

Use smart pointers to handle ktxTexture resources and clean them up.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
To be reverted once the linkage issue for ASTCENC is solved.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
walcht added 17 commits July 10, 2026 18:00
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Prior to this commit, libktx defaulted to using AVX2 on Intel-based
MacOS runners which do not support that SIMD. This caused hard-to-debug
linkage error messages related to ASTCENC whose source files were not
integrated in libktx. For more details, see:
KhronosGroup/KTX-Software#1210

Revert previous temporary commits that were used to figure out
hard-to-locally-reproduce linkage errors with libktx.a.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Remove wrong CMake generator expression. Unsurprisingly, whenever I use
them I mess something up.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Prior to this commit, CMAKE_ARGS was provided as a single string of
"[<-D var=value>...]". This is incorrect as CMAKE_ARGS have to be
provided as a list and not a string.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
* Following different approaches to adding Volume and Cubemap support to
  ktx. Expecting major changes so saving a temporary commit in case
  new approach doesn't work.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
* Removed because std::from_char on MacOS still doesn't support parsing
  of floating point values.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
* CIs with old gcc versions ("oldest" CIs) use an older version of CMake
  than that is required by libktx 5.0.0. This commit attempts to disable
  ktx plugin testing on such old configurations by setting ENABLE_KTX to
  0.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
* libktx v4.3.2 is removed from build_Ktx.cmake and, from now on, only
  libktx v5.0.0 or newer is supported.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
* Add initial documentation about ktx plugin and its exposed attributes

* Add more extensive testing

* Misc cleanups

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@walcht
walcht marked this pull request as ready for review July 22, 2026 17:01
walcht added 2 commits July 22, 2026 19:11
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@walcht

walcht commented Jul 22, 2026

Copy link
Copy Markdown
Author

PR is now ready for initial review. There are still some missing tests but at this point I will wait for initial review then complete missing tests as necessary (there are also a lot of introduced "ktx:<attrib>" attributes. I don't know of any other way to configure the high number of parameters of libktx in ktxoutput).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KTX format

2 participants