Pagination for wlm/stats api #17638

Lindsay-00 · 2025-03-19T22:24:19Z

Description

This change addresses a scalability issue in the existing /_wlm/stats API by introducing a new paginated endpoint: /_list/wlm_stats. The original API returns statistics for all nodes and query groups in a single response, which becomes inefficient and resource-intensive as cluster size grows.

This enhancement is part of the ongoing effort to improve observability and scalability in OpenSearch Workload Management, as described in GitHub Issue #17592.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-03-19T22:46:33Z

❌ Gradle check result for ad612da: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-22T08:51:10Z

❌ Gradle check result for baa30b0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/action/pagination/WlmPaginationStrategy.java

server/src/main/java/org/opensearch/common/Table.java

server/src/main/java/org/opensearch/rest/action/admin/cluster/RestWlmStatsAction.java

server/src/test/java/org/opensearch/action/pagination/DiscoveryNodeMock.java

github-actions · 2025-03-26T06:21:25Z

❌ Gradle check result for 0dd053d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/common/Table.java

server/src/main/java/org/opensearch/action/pagination/WlmPaginationStrategy.java

server/src/main/java/org/opensearch/rest/action/admin/cluster/RestWlmStatsAction.java

server/src/main/java/org/opensearch/action/pagination/SortBy.java

ruai0511 · 2025-03-27T07:09:50Z

Can we provide a better name for this PR and add some description? A single word title is not descriptive

github-actions · 2025-05-31T01:56:04Z

❌ Gradle check result for 29ec21f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Lingxi Chen <[email protected]>

github-actions · 2025-06-02T23:53:16Z

❌ Gradle check result for 35a391b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

jainankitk · 2025-06-03T05:10:25Z

@Lindsay-00 - I can see related test failures:

[Test Result](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/) (14 failures / +12)

    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=wlm_stats/10_basic/Custom size param still returns headers}](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_wlm_stats_10_basic_Custom_size_param_still_returns_headers_/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=wlm_stats/10_basic/Sort by node_id asc does not error}](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_wlm_stats_10_basic_Sort_by_node_id_asc_does_not_error_/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=wlm_stats/10_basic/Invalid sort field returns error}](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_wlm_stats_10_basic_Invalid_sort_field_returns_error_/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=wlm_stats/10_basic/Sort by workload_group_id desc does not error}](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_wlm_stats_10_basic_Sort_by_workload_group_id_desc_does_not_error_/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=wlm_stats/10_basic/Invalid sort order returns error}](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_wlm_stats_10_basic_Invalid_sort_order_returns_error_/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=wlm_stats/10_basic/Sort by node_id asc does not error}](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_wlm_stats_10_basic_Sort_by_node_id_asc_does_not_error__2/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=wlm_stats/10_basic/Invalid sort field returns error}](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_wlm_stats_10_basic_Invalid_sort_field_returns_error__2/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=wlm_stats/10_basic/Sort by workload_group_id desc does not error}](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_wlm_stats_10_basic_Sort_by_workload_group_id_desc_does_not_error__2/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=wlm_stats/10_basic/Invalid sort order returns error}](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_wlm_stats_10_basic_Invalid_sort_order_returns_error__2/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=wlm_stats/10_basic/Invalid sort field returns error}](https://build.ci.opensearch.org/job/gradle-check/58688/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_wlm_stats_10_basic_Invalid_sort_field_returns_error__3/)

Signed-off-by: Lingxi Chen <[email protected]>

github-actions · 2025-06-03T21:44:22Z

✅ Gradle check result for 97d6868: SUCCESS

jainankitk · 2025-06-03T22:21:02Z

@Lindsay-00 - The code coverage for this change is bit low - https://app.codecov.io/gh/opensearch-project/OpenSearch/pull/17638?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=checks&utm_campaign=pr+comments&utm_term=opensearch-project. I am wondering if there is opportunity to improve that by adding few more tests?

Signed-off-by: Lingxi Chen <[email protected]>

github-actions · 2025-06-03T23:53:40Z

❌ Gradle check result for e48e03a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Lingxi Chen <[email protected]>

github-actions · 2025-06-04T01:09:22Z

❌ Gradle check result for bbad32e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Lingxi Chen <[email protected]>

github-actions · 2025-06-04T02:44:28Z

✅ Gradle check result for 0330990: SUCCESS

Signed-off-by: Lingxi Chen <[email protected]>

github-actions · 2025-06-04T04:13:29Z

❌ Gradle check result for c808b69: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Lingxi Chen <[email protected]>

github-actions · 2025-06-04T07:24:10Z

✅ Gradle check result for d2e01fb: SUCCESS

Signed-off-by: Lingxi Chen <[email protected]>

github-actions · 2025-06-04T09:54:49Z

❕ Gradle check result for 4dbd9e4: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

--------- Signed-off-by: Lingxi Chen <[email protected]>

kaushalmahi12 reviewed Mar 24, 2025

View reviewed changes

ruai0511 reviewed Mar 27, 2025

View reviewed changes

opensearch-ci-bot mentioned this pull request May 31, 2025

[AUTOCUT] Gradle Check Flaky Test Report for IndexStatsIT #15836

Open

removed checking if data is changed

35a391b

Signed-off-by: Lingxi Chen <[email protected]>

sgup432 approved these changes Jun 3, 2025

View reviewed changes

sgup432 mentioned this pull request Jun 3, 2025

[Feature Request] Enable the comment creator to resolve their own comment #18418

Open

jainankitk approved these changes Jun 3, 2025

View reviewed changes

fix for failed yaml tests

97d6868

Signed-off-by: Lingxi Chen <[email protected]>

Lingxi Chen added 2 commits June 3, 2025 16:03

adding more tests

f7b1003

Signed-off-by: Lingxi Chen <[email protected]>

more yaml tests

e48e03a

Signed-off-by: Lingxi Chen <[email protected]>

fix failed test

bbad32e

Signed-off-by: Lingxi Chen <[email protected]>

fix yaml

0330990

Signed-off-by: Lingxi Chen <[email protected]>

added uts for restWlmStatsAction

c808b69

Signed-off-by: Lingxi Chen <[email protected]>

fix failed test

d2e01fb

Signed-off-by: Lingxi Chen <[email protected]>

added more uts

4dbd9e4

Signed-off-by: Lingxi Chen <[email protected]>

jainankitk merged commit d2b3ba0 into opensearch-project:main Jun 4, 2025
30 checks passed

Gagan6164 pushed a commit to Gagan6164/OpenSearch that referenced this pull request Jun 8, 2025

Pagination for wlm/stats api (opensearch-project#17638)

e2646a7

--------- Signed-off-by: Lingxi Chen <[email protected]>

Gagan6164 pushed a commit to Gagan6164/OpenSearch that referenced this pull request Jun 8, 2025

Pagination for wlm/stats api (opensearch-project#17638)

cc40abc

--------- Signed-off-by: Lingxi Chen <[email protected]>

rgsriram pushed a commit to rgsriram/OpenSearch that referenced this pull request Jun 9, 2025

Pagination for wlm/stats api (opensearch-project#17638)

c8d413a

--------- Signed-off-by: Lingxi Chen <[email protected]>

abhita pushed a commit to abhita/OpenSearch that referenced this pull request Jun 9, 2025

Pagination for wlm/stats api (opensearch-project#17638)

8d1525c

--------- Signed-off-by: Lingxi Chen <[email protected]>

saimedhi mentioned this pull request Jun 9, 2025

[FEATURE] Add specification for paginated wlm/stats API opensearch-project/opensearch-api-specification#913

Open

Pagination for wlm/stats api #17638

Pagination for wlm/stats api #17638

Uh oh!

Conversation

Lindsay-00 commented Mar 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Mar 19, 2025

Uh oh!

github-actions bot commented Mar 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 26, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ruai0511 commented Mar 27, 2025

Uh oh!

github-actions bot commented May 31, 2025

Uh oh!

github-actions bot commented Jun 2, 2025

Uh oh!

jainankitk commented Jun 3, 2025

Uh oh!

github-actions bot commented Jun 3, 2025

Uh oh!

jainankitk commented Jun 3, 2025

Uh oh!

github-actions bot commented Jun 3, 2025

Uh oh!

github-actions bot commented Jun 4, 2025

Uh oh!

github-actions bot commented Jun 4, 2025

Uh oh!

github-actions bot commented Jun 4, 2025

Uh oh!

github-actions bot commented Jun 4, 2025

Uh oh!

github-actions bot commented Jun 4, 2025

Uh oh!

Uh oh!

Uh oh!

Lindsay-00 commented Mar 19, 2025 •

edited

Loading