๐ Languages: English | ไธญๆ
๐ TL;DR. SAE features are decoded linearly, but their effects on model behavior may depend on mutual interactions. We introduce SAE Additivity Residuals (SAR) to measure the behavioral variation left unexplained by individual feature effects.
๐ Paper (arXiv) ย | ย ๐๏ธ 5-minute tutorial ย | ย ๐งช Examples
Figure 1. Behavioral compositionality in SAE coordinates. (a) Selected decoded contributions are retained or
removed while all other activation content is fixed, and the patched activation is propagated through the frozen upper
layers to obtain
๐ป Click to open the tutorial in your browser
โ๏ธ Try SAR in Colab: Interactive quickstart ยท Use your own SAE
- ๐งฎ An exact interaction decomposition. Under independent feature subset sampling,
$\mathrm{SAR}_r$ equals the normalized energy of interactions above order$r$ . It also bounds coalition ordering error for the best order$r$ approximation. - ๐ Affine controls. Affine readouts give
$\widehat{\mathrm{SAR}}_1$ at numerical zero, at most$1.55\times10^{-13}$ . Real behaviors give a positive$\widehat{\mathrm{SAR}}_1$ in all 57 model, layer and behavior combinations. - ๐งญ SAE coordinates matter. Learned SAE coordinates have lower first order residuals than sum preserving random rotations of the same subspace in all 18 model and behavior settings.
- ๐ค Pairs recover missing structure. Pairwise terms raise the average held out rank fidelity from 0.9782 to 0.9959. They also outperform parameter matched models with random third order terms in all 18 settings.
- ๐๏ธ Steering. For 16 published SAE steering features in Llama-3.1 8B, pairwise modeling raises rank fidelity on MMLU-Pro from 0.5874 to 0.7790.
- ๐ฌ Beyond reconstruction. Across six matched SAEs,
$\widehat{\mathrm{SAR}}_1$ is more strongly associated with sign error and selection regret than reconstruction error. - โก Small code. The estimator is one NumPy file,
sar.py. The GPT-2 demo runs in about a minute on a GPU.
๐ฏ The intervention.
For a prompt
where
๐งฉ SAE Additivity Residuals.
Subsets are drawn by keeping each feature independently with probability
$$\mathrm{SAR}r=\frac{\lVert g_x-\Pi_r g_x\rVert\mu^2}{V},\qquad \Delta_2=\mathrm{SAR}_1-\mathrm{SAR}_2,\qquad \rho_2=\frac{\Delta_2}{\mathrm{SAR}_1}.$$
๐ฒ Estimation.
With
โ Explanation fidelity.
On the held out subsets
| Metric | Definition | Meaning |
|---|---|---|
| Rank fidelity | 1 means the model orders all held out subsets correctly | |
| Ordering error | Wrongly ordered pairs of subsets, weighted by their behavior difference | |
| Sign error | Share of |
The model gets the direction of the change wrong |
| Selection regret | Loss from picking the subset the model ranks first |
Here
๐ข Numerical conventions.
sar.py keeps the conventions of the code behind the paper, so its numbers match the
paper. The estimate uses sar warns when
๐ Rotated coordinates.
To test whether compositionality depends on the SAE basis, we replace the contributions
๐ Explaining a combination of features.
A common way to explain what several features do together is to add up what each one does alone. This is the first order model. We compare it with the pairwise model and with a control that adds the same number of random terms of three features. All models are fit on the same 256 subsets and judged on the same 128 held out subsets. Numbers are averages over 18 model and behavior settings.
| Model | Terms for |
Held out rank fidelity | Ordering error |
|---|---|---|---|
| First order, one effect per feature | 17 | 0.9782 | baseline |
| ๐ Pairwise, one effect per feature and per pair | 137 | โ 0.9959 | โ lower in 18 of 18 settings |
| Random third order control | 137 | โ below pairwise in 18 of 18 settings | โ above pairwise in 18 of 18 settings |
โ๏ธ Evaluating an SAE.
Reconstruction error measures how accurately an SAE represents model activations, but it does not directly
measure whether decoded features have additive downstream effects. For six SAEs at the same Gemma-3 1B
site, we compare how reconstruction error and
| Score | What it measures | With sign error | With selection regret |
|---|---|---|---|
| Reconstruction error |
How well the SAE rebuilds the activation | โ โ0.114 to โ0.013 | โ 0.005 to 0.028 |
| ๐ |
Behavior left unexplained by individual feature effects | โ 0.269 to 0.430 | โ 0.199 to 0.235 |
Clone the repository and install the NumPy core:
git clone https://github.com/OpenSAE/SAE-Behavioral-Compositionality.git
cd SAE-Behavioral-Compositionality
python -m pip install numpyProvide the activation at an SAE site, an SAE encoder and decoder, and a function that scores patched activations:
import numpy as np
from sar_runner import ActivationSite, FunctionalModelAdapter, FunctionalSAE, evaluate_sae
h = np.array([1.5, 1.0, 0.8, 0.6, 0.4, 0.3])
model = FunctionalModelAdapter(
capture=lambda prompt: ActivationSite(activation=h),
score=lambda site, patched: patched.sum(axis=1) + 1.1 * patched[:, 0] * patched[:, 2],
)
sae = FunctionalSAE(encode_fn=lambda x: x, decoder=np.eye(6))
result = evaluate_sae(model, sae, prompt="example", K=6, M=384, n_fit=256)
print("Nonadditivity detected:", result.metrics["sar1"] > 0.01)
print("Pairs explain interactions:", result.metrics["sar2"] < 1e-16)
# Nonadditivity detected: True
# Pairs explain interactions: TrueReplace the example activation, encoder and decoder with your model's SAE site. The scorer must return one scalar per patched activation. evaluate_sae selects the top SAE contributions, evaluates held-out coalitions and returns first-order and pairwise residuals. The estimator alone is sar(g, masks, fit, held) in sar.py.
Core API and real-model adapters ยท Runnable examples ยท Colab notebooks
๐ง Figure 2. Behavioral compositionality varies across behaviors and representations
Prompt median
| Simple copying | Subject verb agreement | Gendered pronoun | CounterFact | IOI | Greater than | |
|---|---|---|---|---|---|---|
| Gemma-3 4B, layer 22 | 0.03668 | 0.03732 | 0.03532 | 0.05275 | 0.04174 | 0.02121 |
| Mistral-7B, layer 24 | 0.02029 | 0.02435 | 0.04939 | 0.05535 | 0.01510 | 0.04159 |
| Llama-3.1 8B, layer 21 | 0.04486 | 0.03070 | 0.08244 | 0.07707 | 0.04245 | 0.05625 |
๐งญ Figure 3. Compositionality depends on coordinate choice
Median
| SAE / rotation | Simple copying | Subject verb agreement | Gendered pronoun | CounterFact | IOI | Greater than |
|---|---|---|---|---|---|---|
| Gemma-3 4B | 0.041 / 0.083 | 0.033 / 0.076 | 0.039 / 0.081 | 0.056 / 0.094 | 0.021 / 0.024 | 0.022 / 0.032 |
| Mistral-7B | 0.023 / 0.058 | 0.022 / 0.120 | 0.024 / 0.141 | 0.055 / 0.133 | 0.016 / 0.038 | 0.065 / 0.066 |
| Llama-3.1 8B | 0.059 / 0.166 | 0.019 / 0.070 | 0.109 / 0.149 | 0.080 / 0.138 | 0.049 / 0.079 | 0.063 / 0.178 |
The selected subspace, the total contribution and the upper layers are the same in each pair. Only the coordinates change, and the SAE coordinates have the lower residual in all 18 settings.
๐ Figure 4. Estimated SAR is associated with first order coalition fidelity
From the lowest to the highest
๐ค Figure 5. Feature pairs recover missing coalition structure
| Average over 18 settings | First order | Pairwise |
|---|---|---|
| Rank fidelity | 0.9782 | 0.9959 |
| Selection regret | 0.0272 | 0.0051 |
Pairwise models also lower ordering error and sign error in all 18 settings, and the gains hold in all 20 repeated splits of the subsets.
๐ช Figures 6 and 7. Estimated compositionality varies across depth
Simple copying at the earliest and latest sampled layers.
| GPT-2 Small | Pythia-70M | Gemma-3 270M | Gemma-3 1B | Gemma-3 4B | Mistral-7B | Llama-3.1 8B | |
|---|---|---|---|---|---|---|---|
| 0.298 โ 0.013 | 0.137 โ 0.0007 | 0.174 โ 0.007 | 0.154 โ 0.015 | 0.112 โ 0.012 | 0.177 โ 0.020 | 0.200 โ 0.256 | |
| 0.34 โ 0.95 | 0.59 โ 1.00 | 0.70 โ 0.99 | 0.70 โ 0.98 | 0.58 โ 0.71 | 0.53 โ 0.79 | 0.54 โ 0.78 |
Across the nine early to late comparisons of Figure 6,
๐๏ธ Table 1. Pairwise modeling better predicts the aggregate steering response
Sixteen published CorrSteer features in Llama-3.1 8B, 1,024 steering combinations, and 1,400 MMLU-Pro questions.
| Approximation | Residual estimate | Rank fidelity | Ordering error |
|---|---|---|---|
| First order | 0.6424 | 0.5874 | 0.1444 |
| Pairwise | 0.3891 | 0.7790 | 0.0606 |
๐ฌ Table 2 and Figure 8. Reconstruction quality and SAR differ in a matched SAE comparison
Prompt level Spearman correlations pooled over six SAEs at Gemma-3 1B layer 17, with 1,800 prompts per behavior.
| Behavior |
|
|
|
|
|---|---|---|---|---|
| Simple copying | โ0.080 | 0.269 | 0.011 | 0.220 |
| Subject verb agreement | โ0.114 | 0.430 | 0.005 | 0.235 |
| IOI | โ0.013 | 0.396 | 0.028 | 0.199 |
๐ Figure index and captions
Captions are taken from the paper.
| Figure | Caption | In the demo |
|---|---|---|
| Figureย 1 | Behavioral compositionality in SAE coordinates. | โ |
| Figureย 2 | Prompt median |
|
| Figureย 3 | Paired first order SAR for learned SAE coordinates and sum preserving random rotations of the same selected activation subspace. | โ panel A |
| Figureย 4 | First order explanation fidelity across |
|
| Figureย 5 | Held out coalition ordering for first order and pairwise approximations. | โ panel D |
| Figureย 6 | Depth profiles for simple copying, CounterFact, and IOI. | โ panels B and C |
| Figureย 7 | Simple copying at the earliest and latest sampled layers across seven model and SAE pipelines. | โ GPT-2 row |
| Figureย 8 | Median prompt level |
|
| Figureย 9 | Behavior scores for four examples under ablations of individual features and feature pairs, together with first order and pairwise predictions when available. |
This project builds on SAE Lens, TransformerLens and Neuronpedia. It uses the open SAEs for GPT-2 Small, Pythia-70M, Gemma Scope 2, Mistral-7B and Llama Scope, the CounterFact, IOI and MMLU-Pro data, and the CorrSteer steering features.
If you use this work, please cite the paper:
@misc{ma2026behavioral,
title = {Behavioral Compositionality of Sparse Autoencoder Representations},
author = {Haotian Ma and Ziye Mei and Frederic Sala},
year = {2026},
eprint = {2610.21261},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.21261}
}








