Add support for search pipeline in search and msearch template

[owaiskazi19]

Description

Add support for search pipeline in msearch template

{"index":"my-nlp-index1"}
{"id":"search_template_1","params":{"play_name":"hello","from":0,"size":1}, "search_pipeline": "my_pipeline2"}
{"index":"my-nlp-index1"}
{"id":"search_template_2","params":{"play_name":"zoo","from":0,"size":1}, "search_pipeline": "my_pipeline1"}

search template

{
  "id": "search_template_2",
  "params": {
    "play_name": "zoo",
    "from": 0,
    "size": 1
  },
  "search_pipeline": "my_pipeline1"
}

Related Issues

Resolves #18508

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 7a5f6eb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 7a5f6eb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 0bdc3bd: SUCCESS

[codecov]

Codecov Report

Attention: Patch coverage is 61.53846% with 10 lines in your changes missing coverage. Please review.

Project coverage is 72.57%. Comparing base (6bf1a6d) to head (1d360cd).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...arch/script/mustache/RestSearchTemplateAction.java	0.00%	3 Missing ⚠️
...nsearch/script/mustache/SearchTemplateRequest.java	82.35%	0 Missing and 3 partials ⚠️
.../script/mustache/SearchTemplateRequestBuilder.java	0.00%	2 Missing ⚠️
...script/mustache/RestMultiSearchTemplateAction.java	66.66%	0 Missing and 1 partial ⚠️
...g/opensearch/action/search/MultiSearchRequest.java	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18564      +/-   ##
============================================
- Coverage     72.75%   72.57%   -0.18%     
+ Complexity    68258    68121     -137     
============================================
  Files          5549     5549              
  Lines        313737   313761      +24     
  Branches      45506    45511       +5     
============================================
- Hits         228250   227710     -540     
- Misses        66919    67483     +564     
  Partials      18568    18568              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[owaiskazi19]

@msfroh @reta @saratvemulapalli @dbwiddis can you review this one whenever possible?

[reta]

@msfroh @reta can you review this one whenever possible?

Sincere apologies @owaiskazi19 , I may get to it next week earliest

[github-actions]

❌ Gradle check result for b33ef49: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[msfroh]

@owaiskazi19 -- for consistency with the _search and _msearch APIs, should we detect if the search_pipeline argument is a string or object? That would let folks include ad hoc pipelines in their msearch template request.

It makes it a bit more complicated, but I think it would ensure that all three APIs behave the same.

Also, do we support search pipelines on regular search template requests? I think if we're fixing msearch template, we need to address regular search templates too.

[dbwiddis]

Well documented issue and good tests that made it clear what the changes are. LGTM!

[owaiskazi19]

for consistency with the _search and _msearch APIs, should we detect if the search_pipeline argument is a string or object? That would let folks include ad hoc pipelines in their msearch template request.

@msfroh sure, we can have a follow up on this PR to support pipeline as object. Need to enhance the parser for that. I will create a follow up issue.

Also, do we support search pipelines on regular search template requests? I think if we're fixing msearch template, we need to address regular search templates too.

That's a good point. I added support of search pipeline in search templates in 92f4fa8

[github-actions]

✅ Gradle check result for 92f4fa8: SUCCESS

[github-actions]

❌ Gradle check result for fd44ee7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for fd44ee7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for fd44ee7: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions]

❕ Gradle check result for fd44ee7: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[msfroh]

Awesome, thanks a lot @owaiskazi19!

[github-actions]

❕ Gradle check result for 1d360cd: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.