README "Convert from .safetensors" flow is broken: convert-ms-to-gguf-bitnet.py KeyError MODEL_ARCH.BITNET_25, and llama-quantize has no I2_S ftype (setup_env.py i2_s conversion path also affected)

[claudioaldrighettiatroos]

Summary

The README section "Convert from .safetensors Checkpoints" (python ./utils/convert-helper-bitnet.py ./models/bitnet-b1.58-2B-4T-bf16) cannot complete on the current main. It fails in two independent places; the second one also breaks the i2_s conversion path of setup_env.py for any .safetensors checkpoint. Reproduced with the official microsoft/bitnet-b1.58-2B-4T-bf16 checkpoint; platform-independent as far as I can tell (found on macOS, but the failing code has no platform branches).

Environment

• microsoft/BitNet at 0b341e5, submodule 3rdparty/llama.cpp at 390c3077, gguf installed from 3rdparty/llama.cpp/gguf-py as done by setup_env.py
• macOS 26.5.2, Apple M2 Pro, Python 3.14.7, numpy 2.5.2, torch 2.14.0, transformers 5.16.1
• build/bin/llama-quantize built from this tree (macOS build fixed as described in setup_env.py on MacBook M2 failed #611 and ggml-cpu.c: src1_cont undeclared → build fails on ARM CPUs with i8mm (Apple M2/M3/M4, Graviton3, ...) #618)

Failure 1: convert-ms-to-gguf-bitnet.py crashes with KeyError: MODEL_ARCH.BITNET_25

huggingface-cli download microsoft/bitnet-b1.58-2B-4T-bf16 --local-dir ./models/bitnet-b1.58-2B-4T-bf16
python ./utils/convert-helper-bitnet.py ./models/bitnet-b1.58-2B-4T-bf16

Preprocessing huggingface checkpoint...        (ok)
Converting to GGUF (f32)...
  File ".../utils/convert-ms-to-gguf-bitnet.py", line 1148, in __init__
    self.gguf = gguf.GGUFWriter(fname_out, gguf.MODEL_ARCH_NAMES[ARCH], endianess=endianess)
KeyError: <MODEL_ARCH.BITNET_25: 83>

Root cause: in 3rdparty/llama.cpp/gguf-py/gguf/constants.py the entries for MODEL_ARCH.BITNET and MODEL_ARCH.BITNET_25 were appended to the wrong dictionary. They sit inside VISION_PROJECTOR_TYPE_NAMES (lines 1141-1142) instead of MODEL_ARCH_NAMES (which ends at line 1129 and only contains BITNET and BITNET_B158). MODEL_ARCH.BITNET_25 is defined in the enum (line 493) and has tensor names (line 3410), but no architecture name, so gguf.MODEL_ARCH_NAMES[MODEL_ARCH.BITNET_25] raises.

Suggested fix: move the two lines into MODEL_ARCH_NAMES.

Failure 2: llama-quantize does not implement the I2_S ftype

Even with Failure 1 fixed, the next step of convert-helper-bitnet.py (and the i2_s branch of setup_env.py::prepare_model(), which does f32 conversion followed by llama-quantize ... I2_S 1) fails:

./build/bin/llama-quantize models/x/ggml-model-f32.gguf models/x/ggml-model-i2_s.gguf I2_S 1
# llama_quantize: invalid ftype 'I2_S'

LLAMA_FTYPE_MOSTLY_I2_S = 41 exists in include/llama.h, but tools/quantize/quantize.cpp has no I2_S entry in QUANT_OPTIONS (it is absent from --help as well) and src/llama-quant.cpp has no handling for it (grep -c I2_S is 0 in both files). So there is currently no way to produce an I2_S GGUF from a .safetensors checkpoint through either documented path. This only goes unnoticed in the main README flow because the pre-quantized BitNet-b1.58-2B-4T-gguf download makes setup_env.py skip conversion.

What does work

python utils/convert-hf-to-gguf-bitnet.py <checkpoint_dir> --outtype i2_s exports I2_S directly (the path described in docs/bitnet-embeddings-i2s-guide.md). The resulting file has the same tensor layout as the official GGUF (210 I2_S tensors, token_embd F16, 121 F32 norms) and, once the FFN activation bug from #588 is fixed, gives the same greedy answers as the bf16 checkpoint in transformers. It would be worth pointing the README to this path, or restoring I2_S in llama-quantize.

Two small related issues:

• The converter writes general.file_type = 40, but in llama.h 40 is Q1_0 and I2_S is 41, so llama-cli reports ftype: Q1_0.
• setup_env.py::gen_code() selects codegen parameters from the model folder name against a hardcoded list and raises NotImplementedError for any other name, so a fine-tuned checkpoint saved under a custom directory cannot be processed without renaming it to e.g. BitNet-b1.58-2B-4T. Reading model_type from config.json would avoid this.