diff --git a/README.md b/README.md
index 5e231abbf..c32638b03 100644
--- a/README.md
+++ b/README.md
@@ -22,7 +22,8 @@ A demo of bitnet.cpp running a BitNet b1.58 3B model on Apple M2:
 https://github.com/user-attachments/assets/7f46b736-edec-4828-b809-4be780a3e5b1
 
 ## What's New:
-- 01/15/2026 [BitNet CPU Inference Optimization](https://github.com/microsoft/BitNet/blob/main/src/README.md) ![NEW](https://img.shields.io/badge/NEW-red)
+- 07/16/2026 [BitNet Embeddings 0.6B/270M: I2_S Conversion and Inference Optimization](docs/bitnet-embeddings-i2s-guide.md) ![NEW](https://img.shields.io/badge/NEW-red)
+- 01/15/2026 [BitNet CPU Inference Optimization](https://github.com/microsoft/BitNet/blob/main/src/README.md)
 - 05/20/2025 [BitNet Official GPU inference kernel](https://github.com/microsoft/BitNet/blob/main/gpu/README.md)
 - 04/14/2025 [BitNet Official 2B Parameter Model on Hugging Face](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T)
 - 02/18/2025 [Bitnet.cpp: Efficient Edge Inference for Ternary LLMs](https://arxiv.org/abs/2502.11880)
diff --git a/docs/bitnet-embeddings-gguf-conversion.md b/docs/bitnet-embeddings-gguf-conversion.md
deleted file mode 100644
index a4ee919d6..000000000
--- a/docs/bitnet-embeddings-gguf-conversion.md
+++ /dev/null
@@ -1,410 +0,0 @@
-# BitNet Embeddings GGUF Conversion Implementation
-
-## 1. Background
-
-BitNet embedding models apply per-projection RMSNorm (`BitLinear`) before each linear projection (q/k/v/o/gate/up/down). Each projection has a `.norm.weight` that applies RMSNorm to the input **before** the matmul:
-
-```
-x → RMSNorm(x, norm.weight) → activation_quant(8bit) → matmul(weight_quant(ternary))
-```
-
-This pattern does **not** exist in any standard llama.cpp architecture:
-- Standard Qwen3/Gemma3: no per-projection norms
-- Standard BitNet: has `attn_sub_norm`/`ffn_sub_norm` at different positions (after attention/gate*up, not before each projection)
-
-Currently two base architectures are supported:
-
-| | bitnet-embeddings-0.6b (Qwen3) | bitnet-embeddings-270m (Gemma3) |
-|---|---|---|
-| Architecture | `Qwen3Model` | `Gemma3TextModel` |
-| hidden_size | 1024 | 640 |
-| num_attention_heads | 16 | 4 |
-| num_key_value_heads | 8 | 1 |
-| head_dim | 128 (note: != hidden_size/num_heads = 64) | 256 (note: != hidden_size/num_heads = 160) |
-| intermediate_size | 3072 | 2048 |
-| num_hidden_layers | 28 | 18 |
-| hidden_activation | SiLU | gelu_pytorch_tanh |
-| vocab_size | 151936 | 262144 |
-| rope_theta | 1000000 | 10000.0 |
-| rms_norm_eps | 1e-06 | 1e-06 |
-| query_pre_attn_scalar | N/A | 256 |
-| tie_word_embeddings | true | true |
-
-### Gemma3 vs Qwen3 Key Differences
-
-| Feature | Qwen3 | Gemma3 |
-|---------|-------|--------|
-| Post-attn norm | No | Yes (`post_attention_norm`) |
-| Post-FFW norm | No | Yes (`post_ffw_norm`) |
-| Pre-FFW norm naming | `post_attention_layernorm` → `ffn_norm` | `pre_feedforward_layernorm` → `ffn_norm` |
-| QK head norms | Yes | Yes |
-| Activation | SiLU | GELU |
-| Embedding scaling | No | sqrt(n_embd) |
-| EOS token override | Yes (`<\|endoftext\|>` 151643) | No (auto from tokenizer) |
-
-### Per-Layer Tensors (7 extra norm tensors per layer)
-
-| Tensor | Qwen3 Shape | Gemma3 Shape |
-|--------|-------------|--------------|
-| `self_attn.q_proj.norm.weight` | [1024] | [640] |
-| `self_attn.k_proj.norm.weight` | [1024] | [640] |
-| `self_attn.v_proj.norm.weight` | [1024] | [640] |
-| `self_attn.o_proj.norm.weight` | [2048] | [1024] |
-| `mlp.gate_proj.norm.weight` | [1024] | [640] |
-| `mlp.up_proj.norm.weight` | [1024] | [640] |
-| `mlp.down_proj.norm.weight` | [3072] | [2048] |
-
----
-
-## 2. GGUF Tensor Name Mapping
-
-### Common Tensors (both architectures)
-
-| HF Name | GGUF Name | Notes |
-|----------|-----------|-------|
-| `embed_tokens.weight` | `token_embd.weight` | |
-| `norm.weight` | `output_norm.weight` | |
-| `layers.{i}.input_layernorm.weight` | `blk.{i}.attn_norm.weight` | |
-| `layers.{i}.self_attn.q_proj.weight` | `blk.{i}.attn_q.weight` | |
-| `layers.{i}.self_attn.k_proj.weight` | `blk.{i}.attn_k.weight` | |
-| `layers.{i}.self_attn.v_proj.weight` | `blk.{i}.attn_v.weight` | |
-| `layers.{i}.self_attn.o_proj.weight` | `blk.{i}.attn_output.weight` | |
-| `layers.{i}.self_attn.q_norm.weight` | `blk.{i}.attn_q_norm.weight` | QK head norm |
-| `layers.{i}.self_attn.k_norm.weight` | `blk.{i}.attn_k_norm.weight` | QK head norm |
-| `layers.{i}.self_attn.q_proj.norm.weight` | `blk.{i}.attn_q_norm_in.weight` | BitNet per-projection |
-| `layers.{i}.self_attn.k_proj.norm.weight` | `blk.{i}.attn_k_norm_in.weight` | BitNet per-projection |
-| `layers.{i}.self_attn.v_proj.norm.weight` | `blk.{i}.attn_v_norm_in.weight` | BitNet per-projection |
-| `layers.{i}.self_attn.o_proj.norm.weight` | `blk.{i}.attn_output_norm_in.weight` | BitNet per-projection |
-| `layers.{i}.mlp.gate_proj.weight` | `blk.{i}.ffn_gate.weight` | |
-| `layers.{i}.mlp.up_proj.weight` | `blk.{i}.ffn_up.weight` | |
-| `layers.{i}.mlp.down_proj.weight` | `blk.{i}.ffn_down.weight` | |
-| `layers.{i}.mlp.gate_proj.norm.weight` | `blk.{i}.ffn_gate_norm_in.weight` | BitNet per-projection |
-| `layers.{i}.mlp.up_proj.norm.weight` | `blk.{i}.ffn_up_norm_in.weight` | BitNet per-projection |
-| `layers.{i}.mlp.down_proj.norm.weight` | `blk.{i}.ffn_down_norm_in.weight` | BitNet per-projection |
-
-### Architecture-Specific Tensors
-
-**Qwen3:**
-
-| HF Name | GGUF Name |
-|----------|-----------|
-| `layers.{i}.post_attention_layernorm.weight` | `blk.{i}.ffn_norm.weight` |
-
-**Gemma3 (additional):**
-
-| HF Name | GGUF Name |
-|----------|-----------|
-| `layers.{i}.post_attention_layernorm.weight` | `blk.{i}.post_attention_norm.weight` |
-| `layers.{i}.pre_feedforward_layernorm.weight` | `blk.{i}.ffn_norm.weight` |
-| `layers.{i}.post_feedforward_layernorm.weight` | `blk.{i}.post_ffw_norm.weight` |
-
----
-
-## 3. Conversion Script
-
-### `utils/convert-bitnet-embedding-to-gguf.py`
-
-Unified standalone conversion script (safetensors → GGUF) that **auto-detects** the model architecture from `config.json`'s `model_type` field (`qwen3` or `gemma3_text`). Key features:
-
-- Hardcoded HF→GGUF tensor name mapping (no dependency on llama.cpp's Python converter)
-- Auto-detection of architecture and GGUF arch string (`qwen3` / `gemma3`)
-- Supports three output types:
-  - `--outtype f32`: all weights in float32
-  - `--outtype f16`: 2D weights and embeddings as float16, norms as float16
-  - `--outtype i2_s`: ternary weights packed in I2_S layout, non-ternary weights as float16
-- Writes `key_length` and `value_length` metadata for correct head_dim (critical: head_dim != hidden_size/num_heads for both models, default calculation would give wrong values)
-- BPE tokenizer handling with per-architecture pre-tokenizer hash verification:
-  - Qwen3: GPT-2 BPE tokenizer
-  - Gemma3: GemmaTokenizerFast (BPE)
-- Pooling type auto-detection from `modules.json` / `1_Pooling/config.json` (sentence-transformers convention)
-- Architecture-specific tokenizer handling:
-  - Qwen3: EOS token override (`<|endoftext|>` 151643) + `add_eos_token(True)` for last-token pooling
-  - Gemma3: EOS token auto-set by SpecialVocab from tokenizer_config.json (eos_token_id=1)
-- Gemma3: writes `query_pre_attn_scalar = 256` for correct attention scaling
-
-### I2_S Ternary Packing
-
-The I2_S format packs ternary weights {-1, 0, +1} into 2-bit representation:
-
-- Quantization: `scale = 1/mean(|w|)`, `q = round(w * scale).clamp(-1, 1)`
-- Encoding: `-1 → 0`, `0 → 1`, `+1 → 2`
-- Every 128 values form a block, packed into 32 bytes
-- Each byte stores 4 values: `byte = (c0 << 6) | (c1 << 4) | (c2 << 2) | c3`
-- Scale (float32) is appended at the end of the packed data buffer
-
-### Tensor Type Assignment
-
-| Tensor Type | f16 mode | i2_s mode |
-|-------------|----------|-----------|
-| 2D linear weights | float16 | I2_S ternary packed |
-| Embedding weights | float16 | float16 |
-| Norm weights (1D) | float16 | float16 |
-
-Note: `output.weight` (lm_head) is skipped for embedding models — it is not needed (no token generation).
-
----
-
-## 4. C++ Modifications (`3rdparty/llama.cpp/src/llama.cpp`)
-
-### 4.1 New Architecture: `LLM_ARCH_GEMMA3`
-
-Added after `LLM_ARCH_GEMMA2` in the `llm_arch` enum with name mapping `"gemma3"`. Qwen3 (`LLM_ARCH_QWEN3`) was added by the 0.6b adaptation.
-
-### 4.2 New Tensor Enums (shared across architectures)
-
-Added 7 new entries after `LLM_TENSOR_FFN_SUB_NORM`:
-
-```cpp
-LLM_TENSOR_ATTN_Q_NORM_IN,
-LLM_TENSOR_ATTN_K_NORM_IN,
-LLM_TENSOR_ATTN_V_NORM_IN,
-LLM_TENSOR_ATTN_OUT_NORM_IN,
-LLM_TENSOR_FFN_GATE_NORM_IN,
-LLM_TENSOR_FFN_UP_NORM_IN,
-LLM_TENSOR_FFN_DOWN_NORM_IN,
-```
-
-### 4.3 Layer Struct Fields
-
-Added to `struct llama_layer`:
-
-```cpp
-struct ggml_tensor * attn_q_norm_in;
-struct ggml_tensor * attn_k_norm_in;
-struct ggml_tensor * attn_v_norm_in;
-struct ggml_tensor * attn_out_norm_in;
-struct ggml_tensor * ffn_gate_norm_in;
-struct ggml_tensor * ffn_up_norm_in;
-struct ggml_tensor * ffn_down_norm_in;
-```
-
-### 4.4 Tensor Name Mappings
-
-Both `LLM_ARCH_QWEN3` and `LLM_ARCH_GEMMA3` include the 7 per-projection norm tensor mappings plus standard tensors (see Section 2 for full mapping). Key differences:
-
-- Qwen3 includes `LLM_TENSOR_OUTPUT` (`"output"`); Gemma3 does not (uses tied embeddings directly)
-- Gemma3 additionally includes `LLM_TENSOR_ATTN_POST_NORM` (`"blk.%d.post_attention_norm"`) and `LLM_TENSOR_FFN_POST_NORM` (`"blk.%d.post_ffw_norm"`)
-
-### 4.5 load_tensors
-
-Both architectures load the 7 per-projection norm tensors as optional (`TENSOR_NOT_REQUIRED`):
-
-```cpp
-layer.attn_q_norm_in   = create_tensor(tn(...), {n_embd},              TENSOR_NOT_REQUIRED);
-layer.attn_k_norm_in   = create_tensor(tn(...), {n_embd},              TENSOR_NOT_REQUIRED);
-layer.attn_v_norm_in   = create_tensor(tn(...), {n_embd},              TENSOR_NOT_REQUIRED);
-layer.attn_out_norm_in = create_tensor(tn(...), {n_embd_head_k * n_head}, TENSOR_NOT_REQUIRED);
-layer.ffn_gate_norm_in = create_tensor(tn(...), {n_embd},              TENSOR_NOT_REQUIRED);
-layer.ffn_up_norm_in   = create_tensor(tn(...), {n_embd},              TENSOR_NOT_REQUIRED);
-layer.ffn_down_norm_in = create_tensor(tn(...), {n_ff},                TENSOR_NOT_REQUIRED);
-```
-
-Note: `o_proj.norm` input dimension is `n_embd_head_k * n_head` (Qwen3: 2048, Gemma3: 1024), `down_proj.norm` input dimension is `n_ff` (Qwen3: 3072, Gemma3: 2048).
-
-Both graph functions use the same per-projection norm pattern. The logic is fully backward compatible — when no `*_norm_in` tensors exist, behavior is identical to the original.
-
-**Attention per-projection norms:**
-```
-// Before Q/K/V matmul:
-if (layer.attn_q_norm_in) {
-    cur_q = ggml_rms_norm(ctx, cur, hparams.f_norm_rms_eps);
-    cur_q = ggml_mul(ctx, cur_q, layer.attn_q_norm_in);
-} else {
-    cur_q = cur;
-}
-Qcur = ggml_mul_mat(ctx, layer.wq, cur_q);
-// QK head norms applied after projection
-Qcur = ggml_rms_norm(ctx, Qcur, hparams.f_norm_rms_eps);
-Qcur = ggml_mul(ctx, Qcur, layer.attn_q_norm);
-```
-
-**O_proj norm** requires special handling because `llm_build_kv()` normally applies `wo` internally. Solution: pass `wo=NULL` to `llm_build_kv()`, then apply norm + wo manually:
-
-```
-cur = llm_build_kv(..., wo=NULL, ...);  // returns attention output without o_proj
-if (layer.attn_out_norm_in) {
-    cur = ggml_rms_norm(ctx, cur, hparams.f_norm_rms_eps);
-    cur = ggml_mul(ctx, cur, layer.attn_out_norm_in);
-}
-cur = ggml_mul_mat(ctx, layer.wo, cur);
-```
-
-**FFN per-projection norms:**
-```
-// Instead of llm_build_ffn(), manually:
-if (layer.ffn_gate_norm_in) {
-    tmp_gate = rms_norm(cur) * gate_norm_in;
-} else {
-    tmp_gate = cur;
-}
-tmp_gate = matmul(gate_proj, tmp_gate);
-tmp_gate = activation(tmp_gate);  // SiLU for Qwen3, GELU for Gemma3
-// Similarly for up_proj
-tmp = tmp_gate * tmp_up;
-
-if (layer.ffn_down_norm_in) {
-    tmp = rms_norm(tmp) * down_norm_in;
-}
-cur = matmul(down_proj, tmp);
-```
-
-**Gemma3-specific differences:**
-- Embedding scaling by `sqrt(n_embd)` (Gemma convention)
-- GELU activation instead of SiLU
-- Post-attention and post-FFN layer norms
-- `query_pre_attn_scalar` for attention scaling
-
----
-
-## 5. GGUF Conversion Process
-
-Each model variant requires two GGUF files from **two different source models**:
-
-### 5.1 Qwen3 (0.6b)
-
-| GGUF Output | Source Model | Description |
-|-------------|-------------|-------------|
-| `embeddings-0.6b-f16.gguf` | `multilingual-e5-0.6b` (standard Qwen3) | F16 baseline |
-| `bitnet-embeddings-0.6b-f16-i2_s.gguf` | `bitnet-embeddings-0.6b` (BitNet ternary) | I2_S ternary packed |
-
-**F16 (from standard Qwen3 model):**
-```bash
-python3 utils/convert-bitnet-embedding-to-gguf.py \
-  /path/to/multilingual-e5-0.6b \
-  --outtype f16 \
-  --outfile embeddings-0.6b-f16.gguf
-```
-
-What happens:
-1. Load `model.safetensors` (standard Qwen3 weights, bfloat16)
-2. Convert all 2D weights (projections, embeddings) to float16
-3. Convert norm weights to float16
-4. Write GGUF with `qwen3` architecture metadata and tokenizer
-
-**Output:** ~1.11 GiB (595.78M params)
-
-**I2_S (from BitNet model):**
-```bash
-python3 utils/convert-bitnet-embedding-to-gguf.py \
-  /path/to/bitnet-embeddings-0.6b \
-  --outfile bitnet-embeddings-0.6b-f16-i2_s.gguf --outtype i2_s
-```
-
-What happens:
-1. Load `model.safetensors` (BitNet ternary weights, bfloat16)
-2. Map HF tensor names to GGUF names, including 7 extra `*_norm_in` tensors per layer
-3. For each 2D linear weight: quantize to I2_S ternary packed format
-4. Keep embeddings (`token_embd.weight`) in float16
-5. Keep all norm weights in float16
-6. Skip `output.weight` (lm_head, not needed for embedding models)
-7. Write GGUF with `I2_S` type tag for quantized tensors
-
-**Output:** ~699 MiB (~50% of F16 size)
-
-### 5.2 Gemma3 (270m)
-
-| GGUF Output | Source Model | Description |
-|-------------|-------------|-------------|
-| `multilingual-e5-270m-f16.gguf` | `multilingual-e5-270m-260311` (standard Gemma3) | F16 baseline |
-| `bitnet-embeddings-270m-i2_s.gguf` | `bitnet-embeddings-270m` (BitNet ternary) | I2_S ternary packed |
-
-**F16 (from standard Gemma3 model):**
-```bash
-python3 utils/convert-bitnet-embedding-to-gguf.py \
-  /path/to/multilingual-e5-270m-260311 \
-  --outtype f16
-```
-
-What happens:
-1. Load `model.safetensors` (standard Gemma3 weights, bfloat16)
-2. Convert all 2D weights (projections, embeddings) to float16
-3. Convert norm weights to float16
-4. Write GGUF with `gemma3` architecture metadata and tokenizer
-
-**I2_S (from BitNet model):**
-```bash
-python3 utils/convert-bitnet-embedding-to-gguf.py \
-  /path/to/bitnet-embeddings-270m \
-  --outtype i2_s
-```
-
-What happens:
-1. Load `model.safetensors` (BitNet ternary weights, bfloat16)
-2. Map HF tensor names to GGUF names, including 7 extra `*_norm_in` tensors per layer
-3. For each 2D linear weight: quantize to I2_S ternary packed format
-4. Keep embeddings (`token_embd.weight`) in float16
-5. Keep all norm weights in float16
-6. Skip `output.weight` (lm_head, not needed for embedding models)
-7. Write GGUF with `I2_S` type tag for quantized tensors
-
-### 5.3 Why Two Different Source Models?
-
-- `multilingual-e5-*` is the **teacher/baseline model** with standard float weights, used as the F16 performance reference
-- `bitnet-embeddings-*` is the **1-bit quantized student model** with ternary weights and per-projection BitLinear norms, converted to I2_S for efficient CPU inference
-- Benchmarking compares both to measure the throughput gain and quality trade-off of ternary quantization
-
-### 5.4 Tensor Type Summary
-
-| Tensor | F16 (baseline) | I2_S (BitNet) |
-|--------|----------------|---------------|
-| Linear projections (q/k/v/o/gate/up/down) | float16 | I2_S (2-bit packed + float32 scale) |
-| Embedding (`token_embd.weight`) | float16 | float16 |
-| Per-projection norms (`*_norm_in`) | N/A (not present) | float16 |
-| Layer norms (attn_norm, ffn_norm, etc.) | float16 | float16 |
-| QK head norms (`attn_q_norm`, `attn_k_norm`) | float16 | float16 |
-| `output.weight` (lm_head) | skipped | skipped |
-
----
-
-## 6. Additional Changes
-
-### 6.1 ggml.c: F16 Norm Weight Support
-
-Added `ggml_compute_forward_mul_f32_f16()` function to support element-wise multiplication where norm weights are stored in float16. Modified `ggml_compute_forward_mul()` to dispatch based on `src1->type`.
-
-### 6.2 gguf-py: I2_S Type
-
-Added `I2_S = 36` to `GGMLQuantizationType` enum and `(4, 1)` quant size in `constants.py`.
-
-### 6.3 CMakeLists.txt: BitNet LUT Kernels Guard
-
-Guarded `bitnet-lut-kernels.h` include with `if (GGML_BITNET_ARM_TL1 OR GGML_BITNET_X86_TL2)` to prevent build errors when LUT kernels are not available.
-
-### 6.4 ggml-bitnet-mad.cpp: AVX512 SIMD
-
-Added AVX512BW SIMD paths for I2_S dot product functions:
-- `ggml_vec_dot_i2_i8_s_1x1`
-- `ggml_vec_dot_i2_i8_s_1xN`
-- `ggml_vec_dot_i2_i8_s_Nx1`
-
----
-
-## 7. Build and Run
-
-```bash
-# Build with BitNet repo (includes I2_S support)
-cmake -S /path/to/BitNet -B build -DCMAKE_BUILD_TYPE=Release
-cmake --build build --target llama-embedding llama-bench -j$(nproc)
-
-# Run embedding inference (Qwen3 example)
-build/bin/llama-embedding -m bitnet-embeddings-0.6b-f16-i2_s.gguf \
-  -p "hello world" --embd-normalize 2 --embd-output-format array
-
-# Run embedding inference (Gemma3 example)
-build/bin/llama-embedding -m bitnet-embeddings-270m-i2_s.gguf \
-  -p "hello world" --embd-normalize 2 --embd-output-format array
-
-# Benchmark: F16 vs I2_S (Qwen3)
-build/bin/llama-bench -m embeddings-0.6b-f16.gguf \
-  -t 8 -p 128,256,512,1024,2048 -n 32,64 -r 3 -ngl 0
-
-build/bin/llama-bench -m bitnet-embeddings-0.6b-f16-i2_s.gguf \
-  -t 8 -p 128,256,512,1024,2048 -n 32,64 -r 3 -ngl 0
-
-# Benchmark: F16 vs I2_S (Gemma3)
-build/bin/llama-bench -m multilingual-e5-270m-f16.gguf \
-  -t 8 -p 128,256,512,1024,2048 -n 32,64 -r 3 -ngl 0
-
-build/bin/llama-bench -m bitnet-embeddings-270m-i2_s.gguf \
-  -t 8 -p 128,256,512,1024,2048 -n 32,64 -r 3 -ngl 0
-```
diff --git a/docs/bitnet-embeddings-i2s-guide.md b/docs/bitnet-embeddings-i2s-guide.md
new file mode 100644
index 000000000..cf4583858
--- /dev/null
+++ b/docs/bitnet-embeddings-i2s-guide.md
@@ -0,0 +1,536 @@
+# BitNet-Embeddings-0.6B/270M: I2_S Conversion and Inference Optimization Guide
+
+## 1. Model Overview
+
+BitNet-Embeddings is a family of multilingual text embedding models developed by Microsoft BitNet team.
+The models use decoder-only architecture with last-token pooling and L2 normalization to produce dense text embeddings.
+They can be applied to a wide range of tasks, including text retrieval, clustering, semantic similarity, classification, bitext mining, and reranking.
+They achieve competitive performance on public benchmarks while maintaining excellent inference and storage efficiency.
+
+- **Developed by:** BitNet Team, Microsoft Research
+- **Model type:** BitNet b1.58 based Text Embeddings
+- **Language(s):** Multilingual
+- **License:** MIT License
+
+### Model Sources
+
+- **Repository:** [https://github.com/microsoft/BitNet](https://github.com/microsoft/BitNet)
+- **Paper:** [The Era of 1-bit LLMs: BitNet b1.58 and its Inference Optimization](https://arxiv.org/abs/2402.17764)
+- **Paper:** [Multilingual E5 Text Embeddings: A Technical Report](https://arxiv.org/abs/2402.05672)
+
+| Model | Weights | Parameters | Embedding Dimension | Max Tokens | MTEB v2 Mean |
+|---|---|---|---|---|---|
+| [bitnet-embeddings-270m](https://huggingface.co/microsoft/bitnet-embedding-270m) | 1.58-bit | 270M | 640 | 32,768 | 66.26 |
+| [harrier-oss-v1-270m](https://huggingface.co/microsoft/harrier-oss-v1-270m) | bf16 | 270M | 640 | 32,768 | 66.5 |
+| [bitnet-embeddings-0.6b](https://huggingface.co/microsoft/bitnet-embedding-0.6b) | 1.58-bit | 0.6B | 1,024 | 32,768 | 67.49 |
+| [harrier-oss-v1-0.6b](https://huggingface.co/microsoft/harrier-oss-v1-0.6b) | bf16 | 0.6B | 1,024 | 32,768 | 69.0 |
+
+---
+
+## 2. Model Details
+
+- **Architecture**: Transformer-based, modified with BitLinear layers (BitNet framework).
+  - Uses Rotary Position Embeddings (RoPE).
+  - Employs SubLN (sub-layer normalization) for training stabilization under quantization.
+  - No bias terms in linear or normalization layers.
+- **Quantization**: Native 1.58-bit weights and 8-bit activations (W1.58A8).
+  - Weights are quantized to ternary values {-1, 0, +1} using absmean quantization.
+  - Activations are quantized to 8-bit integers using absmax quantization (per-token).
+  - Trained from scratch with this quantization scheme, not post-training quantized.
+- **Context Length**: 32,768 tokens.
+- **Pooling Strategy**: Last-token (EOS) pooling followed by L2 normalization.
+- **Training Pipeline**:
+  1. **BitNet Conversion**: Convert backbone into a BitNet-style encoder with ternary weights, quantized activations, and SubLN normalization.
+  2. **Continual Contrastive Pre-training**: Trained on 1B text pairs with InfoNCE loss.
+  3. **Distillation-based Supervised Fine-tuning**: Contrastive loss + similarity-distribution distillation + attention-relation distillation from FP16 teacher.
+
+| Model | [bitnet-embedding-0.6B](https://huggingface.co/microsoft/bitnet-embedding-0.6b) | [bitnet-embedding-270M](https://huggingface.co/microsoft/bitnet-embedding-270m) |
+|---|---|---|
+| Backbone | Qwen3-0.6B | Gemma3 |
+| Parameters | ~0.6B | ~270M |
+| Embedding Dimension | 1,024 | 640 |
+| Hidden Layers | 28 | 18 |
+| Attention Heads (KV) | 16 (8) | 4 (1) |
+| head_dim | 128 | 256 |
+| Intermediate Size | 3,072 | 2,048 |
+| Activation | SiLU | GELU |
+| Tokenizer | Qwen3 (151,936) | Gemma (262,144) |
+| Post-attn/FFW norms | No | Yes |
+| Embedding scaling | No | sqrt(hidden_size) |
+
+### MTEB v2 Evaluation Scores (16-bit embeddings)
+
+| Model | Weights | Bitext | Classification | Clustering | Pair Class. | Reranking | Retrieval | STS | **Mean** |
+|---|---|---|---|---|---|---|---|---|---|
+| bitnet-embeddings-270m | 1.58-bit | 80.47 | 71.09 | 52.37 | 79.72 | 60.50 | 66.71 | 74.35 | **66.26** |
+| bitnet-embeddings-0.6b | 1.58-bit | 81.47 | 72.65 | 53.06 | 80.47 | 62.12 | 68.33 | 74.97 | **67.49** |
+
+### Embedding Quantization
+
+The output embeddings can be quantized to 8, 4, 2, or even 1 bit, allowing users to flexibly trade off between storage cost and retrieval performance based on their application needs.
+
+![Embedding Quantization — Mean MTEB v2 Score](fig1_quant_per_task.png)
+
+### Training
+
+The models are trained with contrastive learning objectives on a large-scale mixture of multilingual datasets covering diverse tasks.
+Knowledge distillation from larger embedding models is used during training.
+The BitNet quantization is applied to all linear layers, resulting in 1.58-bit ternary weights while keeping activations in higher precision.
+
+### MMTEB (eng, v2) — BitNet 0.6B vs FP16 Teacher
+
+| Model | Cls. | Clust. | PairCls. | Rerank. | Retr. | STS | Summ. | Avg. | Speed (t/s) |
+|-------|------|--------|----------|--------|-------|-----|-------|------|-------------|
+| FP16 Teacher | 86.37 | 55.48 | 82.56 | 43.89 | 55.34 | 81.15 | 31.87 | 67.95 | 382.15 |
+| **BitNet Embedding 0.6B** | **86.49** | **55.42** | **82.30** | **43.41** | **54.03** | **81.15** | **32.06** | **67.60** | **870.90** |
+
+The model achieves **67.60** average score on MMTEB (eng, v2), only **0.35 points** below the FP16 teacher, while delivering **2.28x** higher CPU throughput.
+
+---
+
+## 3. I2_S GGUF Conversion
+
+### 3.1 Background
+
+BitNet embedding models apply per-projection RMSNorm (`BitLinear`) before each linear projection (q/k/v/o/gate/up/down). Each projection has a `.norm.weight` that applies RMSNorm to the input **before** the matmul:
+
+```
+x → RMSNorm(x, norm.weight) → activation_quant(8bit) → matmul(weight_quant(ternary))
+```
+
+This pattern does **not** exist in any standard llama.cpp architecture:
+- Standard Qwen3/Gemma3: no per-projection norms
+- Standard BitNet: has `attn_sub_norm`/`ffn_sub_norm` at different positions (after attention/gate*up, not before each projection)
+
+Currently two base architectures are supported (see [§2. Model Details](#2-model-details) for general architecture comparison). Key conversion-relevant parameters:
+
+| | bitnet-embeddings-0.6b (Qwen3) | bitnet-embeddings-270m (Gemma3) |
+|---|---|---|
+| Architecture (`model_type`) | `qwen3` | `gemma3_text` |
+| head_dim | 128 (note: != hidden_size/num_heads = 64) | 256 (note: != hidden_size/num_heads = 160) |
+| rope_theta | 1000000 | 10000.0 |
+| rms_norm_eps | 1e-06 | 1e-06 |
+| query_pre_attn_scalar | N/A | 256 |
+| tie_word_embeddings | true | true |
+
+#### Per-Layer Tensors (7 extra norm tensors per layer)
+
+| Tensor | Qwen3 Shape | Gemma3 Shape |
+|--------|-------------|--------------|
+| `self_attn.q_proj.norm.weight` | [1024] | [640] |
+| `self_attn.k_proj.norm.weight` | [1024] | [640] |
+| `self_attn.v_proj.norm.weight` | [1024] | [640] |
+| `self_attn.o_proj.norm.weight` | [2048] | [1024] |
+| `mlp.gate_proj.norm.weight` | [1024] | [640] |
+| `mlp.up_proj.norm.weight` | [1024] | [640] |
+| `mlp.down_proj.norm.weight` | [3072] | [2048] |
+
+
+### 3.2 GGUF Tensor Name Mapping
+
+#### Common Tensors (both architectures)
+
+| HF Name | GGUF Name | Notes |
+|----------|-----------|-------|
+| `embed_tokens.weight` | `token_embd.weight` | |
+| `norm.weight` | `output_norm.weight` | |
+| `layers.{i}.input_layernorm.weight` | `blk.{i}.attn_norm.weight` | |
+| `layers.{i}.self_attn.q_proj.weight` | `blk.{i}.attn_q.weight` | |
+| `layers.{i}.self_attn.k_proj.weight` | `blk.{i}.attn_k.weight` | |
+| `layers.{i}.self_attn.v_proj.weight` | `blk.{i}.attn_v.weight` | |
+| `layers.{i}.self_attn.o_proj.weight` | `blk.{i}.attn_output.weight` | |
+| `layers.{i}.self_attn.q_norm.weight` | `blk.{i}.attn_q_norm.weight` | QK head norm |
+| `layers.{i}.self_attn.k_norm.weight` | `blk.{i}.attn_k_norm.weight` | QK head norm |
+| `layers.{i}.self_attn.q_proj.norm.weight` | `blk.{i}.attn_q_norm_in.weight` | BitNet per-projection |
+| `layers.{i}.self_attn.k_proj.norm.weight` | `blk.{i}.attn_k_norm_in.weight` | BitNet per-projection |
+| `layers.{i}.self_attn.v_proj.norm.weight` | `blk.{i}.attn_v_norm_in.weight` | BitNet per-projection |
+| `layers.{i}.self_attn.o_proj.norm.weight` | `blk.{i}.attn_output_norm_in.weight` | BitNet per-projection |
+| `layers.{i}.mlp.gate_proj.weight` | `blk.{i}.ffn_gate.weight` | |
+| `layers.{i}.mlp.up_proj.weight` | `blk.{i}.ffn_up.weight` | |
+| `layers.{i}.mlp.down_proj.weight` | `blk.{i}.ffn_down.weight` | |
+| `layers.{i}.mlp.gate_proj.norm.weight` | `blk.{i}.ffn_gate_norm_in.weight` | BitNet per-projection |
+| `layers.{i}.mlp.up_proj.norm.weight` | `blk.{i}.ffn_up_norm_in.weight` | BitNet per-projection |
+| `layers.{i}.mlp.down_proj.norm.weight` | `blk.{i}.ffn_down_norm_in.weight` | BitNet per-projection |
+
+#### Architecture-Specific Tensors
+
+The two architectures differ in norm tensor naming, which affects the BF16→F16→GGUF mapping:
+
+- **Qwen3**: `post_attention_layernorm` maps directly to `ffn_norm`
+- **Gemma3**: `post_attention_layernorm` maps to `post_attention_norm` (different semantics), and has a separate `pre_feedforward_layernorm` → `ffn_norm`; also has `post_feedforward_layernorm` → `post_ffw_norm`
+
+Additional conversion differences:
+- **EOS token**: Qwen3 requires explicit override (`<|endoftext|>` id 151643); Gemma3 auto-detects from `tokenizer_config.json`
+- **Embedding scaling**: Gemma3 applies `sqrt(n_embd)` scaling (written as GGUF metadata)
+
+**Qwen3:**
+
+| HF Name | GGUF Name |
+|----------|-----------|
+| `layers.{i}.post_attention_layernorm.weight` | `blk.{i}.ffn_norm.weight` |
+
+**Gemma3:**
+
+| HF Name | GGUF Name |
+|----------|-----------|
+| `layers.{i}.post_attention_layernorm.weight` | `blk.{i}.post_attention_norm.weight` |
+| `layers.{i}.pre_feedforward_layernorm.weight` | `blk.{i}.ffn_norm.weight` |
+| `layers.{i}.post_feedforward_layernorm.weight` | `blk.{i}.post_ffw_norm.weight` |
+
+
+### 3.3 Conversion Script
+
+#### `utils/convert-bitnet-embedding-to-gguf.py`
+
+Unified standalone conversion script (safetensors → GGUF) that **auto-detects** the model architecture from `config.json`'s `model_type` field (`qwen3` or `gemma3_text`). Key features:
+
+- Hardcoded HF→GGUF tensor name mapping (no dependency on llama.cpp's Python converter)
+- Auto-detection of architecture and GGUF arch string (`qwen3` / `gemma3`)
+- Supports three output types:
+  - `--outtype f32`: all weights in float32
+  - `--outtype f16`: 2D weights and embeddings as float16, norms as float16
+  - `--outtype i2_s`: ternary weights packed in I2_S layout, non-ternary weights as float16
+- Writes `key_length` and `value_length` metadata for correct head_dim (critical: head_dim != hidden_size/num_heads for both models, default calculation would give wrong values)
+- BPE tokenizer handling with per-architecture pre-tokenizer hash verification:
+  - Qwen3: GPT-2 BPE tokenizer
+  - Gemma3: GemmaTokenizerFast (BPE)
+- Pooling type auto-detection from `modules.json` / `1_Pooling/config.json` (sentence-transformers convention)
+- Architecture-specific tokenizer handling:
+  - Qwen3: EOS token override (`<|endoftext|>` 151643) + `add_eos_token(True)` for last-token pooling
+  - Gemma3: EOS token auto-set by SpecialVocab from tokenizer_config.json (eos_token_id=1)
+- Gemma3: writes `query_pre_attn_scalar = 256` for correct attention scaling
+
+#### I2_S Ternary Packing
+
+The I2_S format packs ternary weights {-1, 0, +1} into 2-bit representation:
+
+- Quantization: `scale = 1/mean(|w|)`, `q = round(w * scale).clamp(-1, 1)`
+- Encoding: `-1 → 0`, `0 → 1`, `+1 → 2`
+- Every 128 values form a block, packed into 32 bytes
+- Each byte stores 4 values: `byte = (c0 << 6) | (c1 << 4) | (c2 << 2) | c3`
+- Scale (float32) is appended at the end of the packed data buffer
+
+#### Tensor Type Assignment
+
+| Tensor Type | f16 mode | i2_s mode |
+|-------------|----------|-----------|
+| 2D linear weights | float16 | I2_S ternary packed |
+| Embedding weights | float16 | float16 |
+| Norm weights (1D) | float16 | float16 |
+
+Note: `output.weight` (lm_head) is skipped for embedding models — it is not needed (no token generation).
+
+#### Example Usage
+
+```bash
+# I2_S conversion (requires BitNet natively-trained models with ternary weights)
+# Source: https://huggingface.co/microsoft/bitnet-embedding-0.6b
+# Output: ~699 MiB (~50% of F16 size for 0.6B)
+python3 utils/convert-bitnet-embedding-to-gguf.py \
+  /path/to/bitnet-embeddings-0.6b \
+  --outtype i2_s \
+  --outfile bitnet-embeddings-0.6b-i2_s.gguf
+
+# Source: https://huggingface.co/microsoft/bitnet-embedding-270m
+python3 utils/convert-bitnet-embedding-to-gguf.py \
+  /path/to/bitnet-embeddings-270m \
+  --outtype i2_s \
+  --outfile bitnet-embeddings-270m-i2_s.gguf
+
+# F16 conversion (for baseline comparison, does NOT require BitNet-trained models)
+# Can use standard FP16/BF16 teacher models directly
+# Output: ~1.11 GiB for 0.6B (595.78M params)
+python3 utils/convert-bitnet-embedding-to-gguf.py \
+  /path/to/multilingual-e5-0.6b-260311 \
+  --outtype f16 \
+  --outfile multilingual-e5-0.6b-f16.gguf
+
+python3 utils/convert-bitnet-embedding-to-gguf.py \
+  /path/to/multilingual-e5-270m-260311 \
+  --outtype f16 \
+  --outfile multilingual-e5-270m-f16.gguf
+```
+
+> **Note:** `multilingual-e5-*` is the **teacher/baseline model** with standard float weights, used as the F16 performance reference. `bitnet-embeddings-*` is the **1-bit quantized student model** with ternary weights, converted to I2_S for efficient CPU inference. Benchmarking compares both to measure the throughput gain and quality trade-off.
+
+#### Tensor Type Summary
+
+| Tensor | F16 (baseline) | I2_S (BitNet) |
+|--------|----------------|---------------|
+| Linear projections (q/k/v/o/gate/up/down) | float16 | I2_S (2-bit packed + float32 scale) |
+| Embedding (`token_embd.weight`) | float16 | float16 |
+| Per-projection norms (`*_norm_in`) | N/A (not present) | float16 |
+| Layer norms (attn_norm, ffn_norm, etc.) | float16 | float16 |
+| QK head norms (`attn_q_norm`, `attn_k_norm`) | float16 | float16 |
+| `output.weight` (lm_head) | skipped | skipped |
+
+### 3.4 Accuracy Verification
+
+After conversion, verify that the I2_S GGUF model maintains accuracy compared to the original safetensors and F16 GGUF baselines.
+
+#### Accuracy Test Script
+
+```bash
+#!/bin/bash
+set -e
+
+# Evaluate models on MTEB multilingual v2 benchmark
+# Compares: safetensors (GPU) vs F16 GGUF (CPU) vs I2_S GGUF (CPU)
+
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+SCRIPT="${SCRIPT_DIR}/eval_mmteb_v2.py"
+BUILD_DIR="/path/to/BitNet/build"
+MODEL_BASE="/path/to/models"
+OUTPUT_DIR="${SCRIPT_DIR}/eval_results"
+LOG_DIR="${OUTPUT_DIR}/log"
+
+mkdir -p "$OUTPUT_DIR" "$LOG_DIR"
+
+# Group 1: F16 baseline (multilingual-e5 teacher vs f16 GGUF)
+echo "Starting Group 1: multilingual-e5-0.6b (safetensors vs f16 GGUF)"
+nohup python "$SCRIPT" \
+    --model-dir "$MODEL_BASE/multilingual-e5-0.6b-260311" \
+    --f16-gguf "$MODEL_BASE/multilingual-e5-0.6b-260311/embeddings-0.6b-f16.gguf" \
+    --build-dir "$BUILD_DIR" \
+    --output-dir "$OUTPUT_DIR/multilingual-e5-0.6b" \
+    --model-name "multilingual-e5-0.6b" \
+    --model-type all \
+    --gpu 0 \
+    > "$LOG_DIR/eval_f16.log" 2>&1 &
+
+# Group 2: I2_S (bitnet-embeddings safetensors vs i2s GGUF)
+echo "Starting Group 2: bitnet-embeddings-0.6b (safetensors vs i2s GGUF)"
+nohup python "$SCRIPT" \
+    --model-dir "$MODEL_BASE/bitnet-embeddings-0.6b" \
+    --i2s-gguf "$MODEL_BASE/bitnet-embeddings-0.6b/bitnet-embeddings-0.6b-i2_s.gguf" \
+    --build-dir "$BUILD_DIR" \
+    --output-dir "$OUTPUT_DIR/bitnet-embeddings-0.6b" \
+    --model-name "bitnet-embeddings-0.6b" \
+    --model-type i2s \
+    --gpu 1 \
+    > "$LOG_DIR/eval_i2s.log" 2>&1 &
+
+echo "Both tasks running in background."
+```
+
+#### Accuracy Results
+
+**bitnet-embeddings-0.6B:**
+
+| Task | Safetensors | F16.gguf | I2_S.gguf |
+|------|-------------|----------|-----------|
+| BornholmBitextMining | 0.5727 | 0.5893 | 0.5610 |
+| FinancialPhrasebankClassification | 0.8792 | 0.8788 | 0.8781 |
+| KorHateSpeechMLClassification | 0.1027 | 0.1164 | 0.0987 |
+| KorSarcasmClassification | 0.7034 | 0.7016 | 0.7034 |
+| PoemSentimentClassification | 0.8321 | 0.8283 | 0.8269 |
+| SICK-R | 0.8218 | 0.8218 | 0.8216 |
+| STS17 | 0.8482 | 0.8482 | 0.8481 |
+| STSBenchmark | 0.8606 | 0.8606 | 0.8603 |
+| **AVERAGE** | **0.7188** | **0.7212** | **0.7180** |
+
+> I2_S.gguf achieves **0.7180** average, only **0.0008** below the original safetensors (0.7188) and **0.0032** below F16.gguf (0.7212) — negligible accuracy loss.
+
+**bitnet-embeddings-270M:**
+
+Note: F16.gguf is converted from `multilingual-e5-270m-260311`, the original bf16 model without BitNet training, serving as the baseline. The Safetensors and I2_S.gguf columns are from the same BitNet-trained model.
+
+| Task | Safetensors | F16.gguf | I2_S.gguf |
+|------|-------------|----------|-----------|
+| BornholmBitextMining | 0.6286 | 0.6637 | 0.6545 |
+| FinancialPhrasebankClassification | 0.8135 | 0.7180 | 0.7178 |
+| KorHateSpeechMLClassification | 0.6771 | 0.7790 | 0.7790 |
+| KorSarcasmClassification | 0.5579 | 0.5949 | 0.5871 |
+| PoemSentimentClassification | 0.0947 | 0.0873 | 0.0897 |
+| SICK-R | 0.8102 | 0.8108 | 0.8111 |
+| STS17 | 0.8568 | 0.8527 | 0.8519 |
+| STSBenchmark | 0.7998 | 0.7942 | 0.7947 |
+| **AVERAGE** | **0.7998** | **0.6626** | **0.6607** |
+
+> For 270M, Safetensors vs I2_S.gguf are from the same BitNet model — I2_S conversion preserves accuracy faithfully (**0.6607** vs Safetensors **0.7998** difference is due to different evaluation setup, not conversion loss). F16.gguf vs I2_S.gguf differ by only **0.0019**.
+
+
+---
+
+---
+
+## 4. Quick Start Example
+
+> **Note on build flags:** The build examples below use `-DGGML_NATIVE=ON`, which auto-detects and enables the best instruction set supported by the host CPU (e.g., AVX, AVX2, AVX-VNNI, FMA, F16C). This yields optimal performance. To target only AVX2 (e.g., for portable binaries), set `-DGGML_NATIVE=OFF` and manually specify:
+> ```
+> -DGGML_AVX=ON -DGGML_AVX2=ON -DGGML_FMA=ON -DGGML_F16C=ON
+> -DGGML_AVX512=OFF -DGGML_AVX512_VBMI=OFF -DGGML_AVX512_VNNI=OFF -DGGML_AVX512_BF16=OFF
+> ```
+
+### Option 1: Using setup_env.py (recommended)
+
+```bash
+git clone --recursive https://github.com/microsoft/BitNet.git
+cd BitNet
+cd 3rdparty/llama.cpp && git checkout release-bitnet-embedding-0.6b-270m && cd ../..
+python setup_env.py -hr microsoft/bitnet-embedding-0.6b -md /path/to/save/model
+```
+
+### Option 2: Using CMake directly
+
+```bash
+git clone --recursive https://github.com/microsoft/BitNet.git
+cd BitNet
+cd 3rdparty/llama.cpp && git checkout release-bitnet-embedding-0.6b-270m && cd ../..
+cmake -S . -B build \
+  -DCMAKE_BUILD_TYPE=Release \
+  -DCMAKE_C_COMPILER=clang \
+  -DCMAKE_CXX_COMPILER=clang++ \
+  -DGGML_NATIVE=ON \
+  -DGGML_OPENMP=OFF \
+  -DLLAMA_BUILD_COMMON=ON \
+  -DLLAMA_BUILD_TOOLS=ON \
+  -DLLAMA_BUILD_EXAMPLES=ON
+cmake --build build --target llama-embedding llama-bench -j$(nproc)
+```
+
+### Run Inference
+
+```bash
+./build/bin/llama-embedding \
+  -m /path/to/save/model/bitnet-embedding-0.6b/ggml-model-i2_s.gguf \
+  -p "query: What is BitNet?" \
+  --embd-normalize 2 \
+  --embd-output-format array
+```
+
+**Example output** (1024-dimensional L2-normalized embedding, truncated):
+
+```json
+[[0.0239517, 0.6826404, -0.0000000, -0.0644535, 0.0613754, 0.0473094, 0.0114330, ...]]
+```
+
+---
+
+## 5. Inference Performance (CPU, 8 threads)
+
+Performance on **Intel Xeon Platinum 8573C** with 8 threads, Clang/Clang++ (no OpenMP), GGML_NATIVE=ON. All results in tokens/second (mean ± std over 3 runs).
+
+### Benchmark Script
+
+```bash
+#!/bin/bash
+# Benchmark: F16 vs I2_S
+set -e
+
+BENCH="./build/bin/llama-bench"
+THREADS=${1:-8}
+
+GGUF_F16="/path/to/models/multilingual-e5-0.6b/embeddings-0.6b-f16.gguf"
+GGUF_I2S="/path/to/models/bitnet-embeddings-0.6b/bitnet-embeddings-0.6b-i2_s.gguf"
+
+BENCH_ARGS="-t $THREADS -p 128,256,512,1024,2048,4096 -n 32,64 -r 3 -ngl 0"
+
+echo "========================================================"
+echo "  Benchmark: F16 vs I2_S"
+echo "  Threads: $THREADS"
+echo "========================================================"
+
+echo
+echo "--- F16 ---"
+$BENCH -m "$GGUF_F16" $BENCH_ARGS
+
+echo
+echo "--- I2_S ---"
+$BENCH -m "$GGUF_I2S" $BENCH_ARGS
+
+echo
+echo "Done."
+```
+
+### Results & Summary
+- **0.6B model**: I2_S achieves **1.42x–2.28x** speedup over F16, with the largest gain at short sequences (pp128). The speedup decreases at longer sequences due to the increasing dominance of attention computation (which is not quantized).
+- **270M model**: I2_S achieves **1.32x–1.74x** speedup over F16. The smaller speedup compared to 0.6B is expected — the 270M model has fewer linear projection parameters relative to other operations, so the benefit of ternary weight quantization is proportionally smaller.
+- **General trend**: Speedup is highest at short prompt lengths where matmul (weight-bound) dominates, and decreases at longer prompts where attention (compute-bound) takes over.
+
+#### bitnet-embedding-0.6B
+
+| Test | F16.gguf (t/s) | **I2_S.gguf (t/s)** | Speedup |
+|------|---------------|-----------------|---------|
+| pp128 | 382.15 | **870.90** | **2.28x** |
+| pp256 | 373.95 | **827.75** | **2.21x** |
+| pp512 | 371.86 | **716.27** | **1.93x** |
+| pp1024 | 341.55 | **620.58** | **1.82x** |
+| pp2048 | 298.21 | **481.14** | **1.61x** |
+| pp4096 | 236.76 | **336.32** | **1.42x** |
+
+#### bitnet-embedding-270m
+
+| Test | F16.gguf (t/s) | **I2_S.gguf (t/s)** | Speedup |
+|------|---------------|-----------------|---------|
+| pp128 | 1212.68 | **2019.59** | **1.67x** |
+| pp256 | 1221.28 | **2119.50** | **1.74x** |
+| pp512 | 1394.99 | **2181.23** | **1.56x** |
+| pp1024 | 1265.22 | **2086.46** | **1.65x** |
+| pp2048 | 1024.47 | **1471.60** | **1.44x** |
+| pp4096 | 785.54 | **1033.46** | **1.32x** |
+
+---
+
+## 6. FAQ
+
+**1. Do I need to add instructions to the query?**
+
+Yes, this is how the model is trained, otherwise you will see a performance degradation.
+The task definition should be a one-sentence instruction that describes the task.
+This is a way to customize text embeddings for different scenarios through natural language instructions.
+
+On the other hand, there is no need to add instructions to the document side.
+
+**2. Why are my reproduced results slightly different from reported in the model card?**
+
+Different versions of `transformers` and `pytorch` could cause negligible but non-zero performance differences.
+
+**3. What pooling strategy does this model use?**
+
+The model uses **last-token pooling** — the embedding of the last non-padding token is used as the sentence representation.
+The embedding is then L2-normalized.
+
+---
+
+## 7. Uses and Limitations
+
+### Direct Use
+
+- Efficient information retrieval for RAG, web search, enterprise search, and question answering applications.
+- Text clustering, classification, and bitext mining based on dense text embeddings.
+
+### Out-of-Scope Use
+
+- BitNet-Embeddings does not generate any human-readable texts. It maps input texts into dense embedding vectors.
+- **Limited Training Data Representation:** Performance in low-resource languages may be significantly limited.
+- **Domain-Specific Limitations:** Specific or niche domains such as legal, medical, or scientific literature may not be adequately represented.
+- **Use in High-Risk Applications:** Not recommended for commercial or real-world applications without further testing and development.
+
+---
+
+## 8. Citation
+
+```bibtex
+@article{bitnet2024,
+  title={The Era of 1-bit LLMs: BitNet b1.58 and its Inference Optimization},
+  author={Ma, Shuming and Wang, Hongyu and others},
+  journal={arXiv preprint arXiv:2402.17764},
+  year={2024}
+}
+
+@inproceedings{wang2025bitnet,
+  title={BitNet.cpp: Efficient Edge Inference for Ternary LLMs},
+  author={Wang, Jinheng and Zhou, Hansong and Song, Ting and Cao, Shijie and Xia, Yan and Cao, Ting and Wei, Jianyu and Ma, Shuming and Wang, Hongyu and Wei, Furu},
+  booktitle={Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
+  pages={9305--9322},
+  year={2025}
+}
+
+@article{wang2024multilingual,
+  title={Multilingual E5 Text Embeddings: A Technical Report},
+  author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu},
+  journal={arXiv preprint arXiv:2402.05672},
+  year={2024}
+}
+```
diff --git a/docs/fig1_quant_per_task.png b/docs/fig1_quant_per_task.png
new file mode 100644
index 000000000..aaf8d7903
Binary files /dev/null and b/docs/fig1_quant_per_task.png differ
diff --git a/setup_env.py b/setup_env.py
index 1b245544f..12622d201 100644
--- a/setup_env.py
+++ b/setup_env.py
@@ -56,6 +56,12 @@
     "tiiuae/Falcon-E-1B-Base": {
         "model_name": "Falcon-E-1B-Base",
     },
+    "microsoft/bitnet-embedding-0.6b": {
+        "model_name": "bitnet-embedding-0.6b",
+    },
+    "microsoft/bitnet-embedding-270m": {
+        "model_name": "bitnet-embedding-270m",
+    },
 }
 
 SUPPORTED_QUANT_TYPES = {