Description
Is your feature request related to a problem? Please describe
To support star tree index in OpenSearch, we need changes in the new composite codec [https://github.com//issues/14387] to build star tree index during merge flows.
Describe the solution you'd like
We will enhance the custom DocValuesFormat to support star tree creation during merge. We will override the merge method and merge the star tree indices across the segments.
The idea is to build the star tree from the existing star tree doc values from each segment instead of again building everything from scratch using segment documents.
We will basically merge all the documents which are not star documents [ essentially documents with values for all dimensions ] and we will build star tree and then we will derive star documents again from the tree.
This method optimizes the performance and resource utilization greatly and also ensures that newly created star tree documents during merge still adheres to 'maxLeafDocs' configuration.
Related component
Indexing:Performance
Describe alternatives you've considered
No response
Additional context
No response