voyage-multimodal-3.5 (video) support

[fzowl]

voyage-multimodal-3.5 (video) support

[bilgeyucel]

Is the multimodal functionality supported by any embedders in the voyage-embedders-haystack package? If so, please use those components

[fzowl]

@bilgeyucel Yes, now multimodal embeddings are supported, so i updated the PR. Can you please take a look?

[bilgeyucel]

Hi @fzowl, thanks for the PR! I only have one comment, and it applies to most of the code examples you shared. It looks like all embedder components act as generator components, generating natural language outputs. Can you also use these models just create embeddings?

[bilgeyucel]

Is this an image embedder or a generative model? It looks like it acts as a generative model (VLMs) 🤔
If that's the case, it's worth renaming it as a VoyageMultimodalChatGenerator with a similar API to this:

from haystack.dataclasses import ImageContent, ChatMessage
from haystack.components.generators.chat import OpenAIChatGenerator

image_content = ImageContent.from_file_path("image.jpg")
user_message = ChatMessage.from_user(content_parts=["Describe the image in short.", image_content])
llm = VoyageMultimodalChatGenerator(model="voyage-multimodal-3.5")

You can check OpenAIChatGenerator for details

[fzowl]

@bilgeyucel Thanks for catching that! You're right that the text strings were misleading.

To clarify: voyage-multimodal-3.5 is an embedding model — it returns float vectors (list[list[float]]), not generated text. The text strings in multimodal inputs are content that gets co-embedded with images/videos into a shared vector space for semantic similarity search. They are not prompts or instructions to the model.

I've updated the three confusing example strings:

• "What is in this image?" → "A sunset over the ocean" (descriptive content, not a question)
• "Describe this image:" → "Product photo for online store" (label, not an instruction)
• "Describe this video:" → "Machine learning tutorial" (topic description, not a prompt)

The other examples ("Document about machine learning", "Technical diagram") already read as descriptive content so those are unchanged.