[Bug Report] TransformerBridge.run_with_cache ignores prepend_bos=False for string inputs

[emerardd]

Describe the bug

TransformerBridge.run_with_cache() ignores prepend_bos=False when its input is a single string. The cached forward therefore runs on a different token sequence from a direct forward with the same arguments.

This is silent: the call succeeds, but the logits and every position-indexed cached activation include an extra BOS position.

Code example

On current dev-4.x (7db5f8dc):

import torch

from transformer_lens.model_bridge import TransformerBridge

bridge = TransformerBridge.boot_transformers("gpt2", device="cpu")
text = "Hello"

tokens = bridge.to_tokens(text, prepend_bos=False)

with torch.no_grad():
    direct_logits = bridge(text, prepend_bos=False)
    cached_logits, cache = bridge.run_with_cache(
        text,
        prepend_bos=False,
    )

print(tokens.shape)
print(direct_logits.shape)
print(cached_logits.shape)
print(cache["hook_embed"].shape)

Observed:

torch.Size([1, 1])
torch.Size([1, 1, 50257])
torch.Size([1, 2, 50257])
torch.Size([1, 2, 768])

I also reproduced this without downloading a checkpoint using a tiny TransformerBridge.boot_native() model and an offline tokenizer:

to_tokens(prepend_bos=False): [[7]]
to_tokens(prepend_bos=True):  [[0, 7]]
direct forward shape:         (1, 1, 8)
run_with_cache shape:         (1, 2, 8)
cached hook_embed shape:      (1, 2, 8)

The direct and cached paths should use the same token sequence when given the same string and prepend_bos value.

The immediate cause is the eager string tokenization in BridgeCore.run_with_cache():

input_ids = self.to_tokens(processed_args[0])

and the equivalent kwargs["input"] branch. These calls do not forward the prepend_bos value from kwargs. The original value is later passed to forward(), but by then the input is already a token tensor, so it can no longer affect tokenization.

This can shift token positions in attention patterns, residual streams, activation patching, and attribution analyses. It can also produce a double BOS for preformatted chat-template text.

A fix should make the eager tokenization path consume and forward its tokenization options, including both prepend_bos and padding_side, and add parity coverage between direct and cached forwards for prepend_bos=True and False.

System Info

• Installed from source
• OS: Windows
• Python: 3.12.10
• Branch/commit: dev-4.x at 7db5f8dc
• Reproduction uses CPU and does not require model downloads in the boot_native variant

Additional context

This is related to the BOS contract addressed for generation in #1439, but that PR did not update run_with_cache(). The older #990 changed this path to call self.to_tokens(), but did not forward tokenization options.

Checklist

• I have checked that there is no similar issue in the repo (required)

[emerardd]

I plan to work on this. My proposed scope is to make run_with_cache() consume and forward prepend_bos before converting scalar string input to a tensor, add direct-vs-cached parity coverage for both BOS settings, and preserve the existing tensor/list and driver-aware device behavior. After inspecting the shared BridgeCore contract, I am keeping this PR focused on the demonstrated prepend_bos bug: padding_side is not a common tokenization option across TransformerBridge and RemoteBridge and has separate existing behavior.