Skip to content

SOLR-18136: Fix ArrayStoreException when combining rerank with sort under multi-threaded segment-parallel search#4164

Open
limingnihao wants to merge 1 commit intoapache:mainfrom
limingnihao:SOLR-18136-rerank-multithreaded-arraystoreexception
Open

SOLR-18136: Fix ArrayStoreException when combining rerank with sort under multi-threaded segment-parallel search#4164
limingnihao wants to merge 1 commit intoapache:mainfrom
limingnihao:SOLR-18136-rerank-multithreaded-arraystoreexception

Conversation

@limingnihao
Copy link
Contributor

https://issues.apache.org/jira/browse/SOLR-18136

Description

When multi-threaded segment-parallel search is enabled (indexSearcherExecutorThreads > 0 and multiThreaded=true) and a query uses both reranking and a sort, an ArrayStoreException is thrown during the merge phase if some Lucene slices have matching documents and others do not.

java.lang.ArrayStoreException: arraycopy: element type mismatch: can not cast one of the elements
of org.apache.lucene.search.TopDocs[] to the type of the destination array,
org.apache.lucene.search.TopFieldDocs

Solution

The root cause is in MultiThreadedSearcher.TopDocsCM.reduce(). It decides how to merge TopDocs from different slices based solely on the type of topDocs[0]:

if (topDocs[0] instanceof TopFieldDocs) {
    TopFieldDocs[] topFieldDocs = Arrays.copyOf(topDocs, topDocs.length, TopFieldDocs[].class);

When ReRankCollector is used with a sort, empty-result slices return TopFieldDocs (from the underlying TopFieldCollector), while non-empty slices return plain TopDocs (after ReRankCollector.toRescoredDocs() wraps results with new TopDocs(...), discarding the TopFieldDocs subtype). If an empty-result slice is first in the array, Arrays.copyOf(..., TopFieldDocs[].class) fails because the array contains mixed types.
The fix checks all elements instead of only topDocs[0]:

if (Arrays.stream(topDocs).allMatch(td -> td instanceof TopFieldDocs))

When not all elements are TopFieldDocs, the code falls through to TopDocs.merge(), which handles mixed types correctly.
This fix was developed with help from an AI coding assistant (Cursor with Claude).

Tests

Added TestMultiThreadedSearcher.testReRankWithMultiThreadedSearch() which:

  • Uses NoMergePolicyFactory to prevent segment merges and guarantee > 5 segments
  • Configures NodeConfig with indexSearcherExecutorThreads=4 to enable parallel search
  • Creates 7 non-matching segments before 1 matching segment, ensuring different Lucene slices see different result counts
  • Exercises the full code path: SolrIndexSearcher.getDocListNC() → MultiThreadedSearcher.searchCollectorManagers() → Lucene parallel search(Query, CollectorManager) → TopDocsCM.reduce()
  • Without the fix, the test fails with the exact same ArrayStoreException seen in production

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide
  • I have added a changelog entry for my change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant