[FLINK-AGENTS-524] Add Amazon OpenSearch and S3 Vectors vector store integrations by avichaym · Pull Request #533 · apache/flink-agents

avichaym · 2026-02-10T16:27:13Z

Linked issue: #524

Depends on #534 — please merge that first.

Purpose of change

Add Amazon OpenSearch and S3 Vectors as vector store providers.

OpenSearchVectorStore — Supports Serverless (AOSS) and Service domains, IAM/basic auth, implements CollectionManageableVectorStore for Long-Term Memory, KNN search with filter support, chunked bulk writes
S3VectorsVectorStore — S3 Vectors SDK, PutVectors chunked at 500 (API limit)

Both override add() for batch embedding optimization.

New modules: integrations/vector-stores/opensearch/, integrations/vector-stores/s3vectors/

Tests

Unit tests: OpenSearch (4), S3 Vectors (2)
Integration tests gated by env vars (OPENSEARCH_ENDPOINT, S3V_BUCKET): collection CRUD, document CRUD, filtered query
End-to-end validated with RAG and Long-Term Memory demos against real OpenSearch domain and S3 Vectors bucket

API

No public API changes. New integration modules only.

Documentation

doc-needed
doc-not-needed
doc-included

github-actions · 2026-02-10T16:38:19Z

@avichaym Please add the following content to your PR description and select a checkbox:

- [ ] `doc-needed` 
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-included`

wenjin272

Hi, @avichaym, thanks for your work. LGTM, just a few minor comments.

wenjin272 · 2026-02-26T08:08:36Z

...java/org/apache/flink/agents/integrations/vectorstores/opensearch/OpenSearchVectorStore.java

+     * Batch-embeds all documents in a single call, then delegates to addEmbedding.
+     *
+     * <p>TODO: This batch embedding logic is duplicated in S3VectorsVectorStore. Consider
+     * extracting to BaseVectorStore in a follow-up (would also benefit ElasticsearchVectorStore).


+1 for implementing this batch embedding logic in BaseVectorStore directly.

Agreed. Will submit as a follow-up PR.

wenjin272 · 2026-02-26T09:02:11Z

...java/org/apache/flink/agents/integrations/vectorstores/opensearch/OpenSearchVectorStore.java

+
+        this.index = descriptor.getArgument("index");
+        if (this.index == null || this.index.isBlank()) {
+            throw new IllegalArgumentException("index is required for OpenSearchVectorStore");


Could index be null but indicate index in each operation?

Good point. Made index optional — when null, operations use the collection parameter passed at call time. This supports use cases where the index is determined per-operation (e.g., Long-Term Memory
collection management)

wenjin272 · 2026-02-26T09:08:20Z

...n/java/org/apache/flink/agents/integrations/vectorstores/s3vectors/S3VectorsVectorStore.java

+            @Nullable List<String> ids, @Nullable String collection, Map<String, Object> extraArgs)
+            throws IOException {
+        if (ids == null || ids.isEmpty()) {
+            return;


In current design, if ids is not provided, vector store should get/delete all the documents in the collection. The behavior of s3vectors is inconsistent, we need throw exception when ids is not provided for s3vectors or emphasize this point in the documentation.

Fixed. Changed get() and delete() to throw UnsupportedOperationException when ids is null. S3 Vectors does have a ListVectors API that could support this, but it's paginated (max 1000/page) and a full list+delete-all could be expensive on large indexes.
I think throwing is the safer default, but can also implement in this or a follow-up .

wenjin272 · 2026-02-26T09:18:16Z

...java/org/apache/flink/agents/integrations/vectorstores/opensearch/OpenSearchVectorStore.java

+            body.put("size", ids.size());
+            return parseHits(executeRequest("POST", "/" + idx + "/_search", body.toString()));
+        }
+        int limit = 10000;


Maybe a static variable is better?

Fixed. Extracted to private static final int DEFAULT_GET_LIMIT = 10000

wenjin272 · 2026-02-26T09:24:34Z

...n/java/org/apache/flink/agents/integrations/vectorstores/s3vectors/S3VectorsVectorStore.java

+        this.client =
+                S3VectorsClient.builder()
+                        .region(Region.of(regionStr != null ? regionStr : "us-east-1"))
+                        .credentialsProvider(DefaultCredentialsProvider.create())


According to https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html#create(), create is deprecated, builder().build() is better.

Fixed in both OpenSearchVectorStore and S3VectorsVectorStore.

wenjin272

Thanks for addressing my comments. Overall looks good to me, just some comments about the retry parameters. Besides, there are some code-style questions.

wenjin272 · 2026-03-03T03:09:06Z

...java/org/apache/flink/agents/integrations/vectorstores/opensearch/OpenSearchVectorStore.java

+
+        this.retryExecutor =
+                RetryExecutor.builder()
+                        .maxRetries(5)


Should this be configuable?

wenjin272 · 2026-03-03T03:09:25Z

...n/java/org/apache/flink/agents/integrations/vectorstores/s3vectors/S3VectorsVectorStore.java

+
+        this.retryExecutor =
+                RetryExecutor.builder()
+                        .maxRetries(5)


…integrations - BedrockChatModelConnection: Converse API with native tool calling, SigV4 auth - BedrockChatModelSetup: model, temperature, max_tokens configuration - BedrockEmbeddingModelConnection: Titan Text Embeddings V2 with parallel batch embedding - BedrockEmbeddingModelSetup: model, dimensions configuration - Retry via unified RetryExecutor with Bedrock-specific retryable predicate - stripMarkdownFences for non-tool-call responses (lossless fence stripping only)

…integrations - OpenSearchVectorStore: supports Serverless (AOSS) and Service domains, IAM and basic auth, SigV4 signing, bulk indexing, collection management for long-term memory - S3VectorsVectorStore: PutVectors/QueryVectors/GetVectors/DeleteVectors, batched puts (500/request limit) - Retry via unified RetryExecutor with service-specific retryable predicates

github-actions bot added priority/major Default priority of the PR or issue. fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. labels Feb 10, 2026

avichaym changed the title ~~[FLINK-AGENTS-523] Add Amazon Bedrock chat model and embedding model integrations~~ [FLINK-AGENTS-524] Add Amazon OpenSearch and S3 Vectors vector store integrations Feb 10, 2026

avichaym force-pushed the feature/aws-vector-stores branch from 56b9836 to 9f4f768 Compare February 10, 2026 16:37

github-actions bot added the doc-label-missing The Bot applies this label either because none or multiple labels were provided. label Feb 10, 2026

github-actions bot added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing The Bot applies this label either because none or multiple labels were provided. labels Feb 10, 2026

wenjin272 added doc-needed Your PR changes impact docs. and removed doc-not-needed Your PR changes do not impact docs labels Feb 12, 2026

avichaym force-pushed the feature/aws-vector-stores branch 2 times, most recently from c492b13 to dd77d50 Compare February 19, 2026 09:08

xintongsong requested a review from wenjin272 February 24, 2026 09:31

wenjin272 reviewed Feb 26, 2026

View reviewed changes

avichaym force-pushed the feature/aws-vector-stores branch from 9c6d61e to 900d687 Compare March 2, 2026 12:30

wenjin272 reviewed Mar 3, 2026

View reviewed changes

avichaym force-pushed the feature/aws-vector-stores branch from 900d687 to 8242655 Compare March 3, 2026 07:37

Avichay Marciano added 2 commits March 3, 2026 09:54

avichaym force-pushed the feature/aws-vector-stores branch from 8242655 to 6e10664 Compare March 3, 2026 07:55

Conversation

avichaym commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose of change

Tests

API

Documentation

Uh oh!

github-actions bot commented Feb 10, 2026

Uh oh!

wenjin272 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wenjin272 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

avichaym commented Feb 10, 2026 •

edited

Loading