Fix Gemma reranker imports with Transformers v5

[Gui-Yue]

Summary

This PR fixes a remaining Transformers v5 import compatibility issue in the Gemma decoder-only reranker.

FlagEmbedding already has Transformers v5 compatibility work in #1563, and #1571 is addressing tokenizer API changes in reranker inference. This PR is complementary to both: it handles an import-time failure caused by gemma_model.py importing private Gemma2 docstring constants from transformers.models.gemma2.modeling_gemma2.

In Transformers v5, GEMMA2_START_DOCSTRING and GEMMA2_INPUTS_DOCSTRING are no longer available from that private module path. They are only used for generated docstrings, so falling back to empty strings preserves importability without changing runtime model behavior.

Changes

• Make GEMMA2_START_DOCSTRING and GEMMA2_INPUTS_DOCSTRING optional in FlagEmbedding/inference/reranker/decoder_only/models/gemma_model.py.
• Add an import test for CostWiseGemmaForCausalLM in tests/test_imports_v5.py.

Reproduction

I reproduced the original import failure in a temporary local environment with:

• Python 3.13
• torch 2.12.0+cpu
• transformers 5.8.1

The pre-fix import statement used by FlagEmbedding/inference/reranker/decoder_only/models/gemma_model.py fails with Transformers v5:

from transformers.models.gemma2.modeling_gemma2 import (
    GEMMA2_START_DOCSTRING,
    GEMMA2_INPUTS_DOCSTRING,
)

Output:

torch 2.12.0+cpu
transformers 5.8.1
old import: FAILED
ImportError: cannot import name 'GEMMA2_START_DOCSTRING' from 'transformers.models.gemma2.modeling_gemma2'

These symbols are only used for generated docstrings, and they are not present in the Transformers v5 modeling_gemma2.py module.

Verification

After this change, the Gemma reranker module imports successfully in the same environment:

from FlagEmbedding.inference.reranker.decoder_only.models.gemma_model import (
    CostWiseGemmaForCausalLM,
)

Output:

torch 2.12.0+cpu
transformers 5.8.1
fixed import: OK
CostWiseGemmaForCausalLM

Top-level import also works:

import FlagEmbedding

Output:

top-level import: OK

I also ran the v5 import test file:

PYTHONPATH=$PWD python -m pytest tests/test_imports_v5.py -q

Output:

5 passed, 2 warnings in 3.40s

Related PRs

• feat: add compatibility with transformers V5 #1563 fixed the is_torch_fx_available compatibility path for Transformers v5.
• fix(reranker): support transformers v5 tokenizer API changes #1571 addresses Transformers v5 tokenizer API changes in reranker inference.
• This PR fixes a separate remaining import-time issue in the Gemma decoder-only reranker caused by docstring-only private constants removed from Transformers v5.

Related Issues

• cannot import name 'Gemma2FlashAttention2' from 'transformers.models.gemma2.modeling_gemma2' #1356 is directly related. It reports an import failure from FlagEmbedding/inference/reranker/decoder_only/models/gemma_model.py after upgrading Transformers, caused by private Gemma2 symbols imported from transformers.models.gemma2.modeling_gemma2.
• huggingface transformers has removed the is_torch_fx_available function #1561 is related background for the same class of Transformers v5 compatibility issue. It covered the removed is_torch_fx_available API and was resolved by feat: add compatibility with transformers V5 #1563.
• Can flagembedding dependancy on transformers upgrade to newer ones? #1266 is broader context asking whether FlagEmbedding can support newer Transformers versions.