Created an API to fetch remote store metadata

[Sukriti0717]

Added Remote Store Metadata API to fetch segment and translog metadata

Description

This PR adds a new cluster-level API that enables users to retrieve remote store metadata, including segment and translog details, for specific indices and shards. This functionality enhances observability and debugging capabilities for indices backed by remote storage.

Key Changes

1. Transport Layer: TransportRemoteStoreMetadataAction:

• Executes on the Data node.
• Collects and parses segment and translog metadata files for each shard.
• Handles shard-level errors gracefully.

2. Core Action Classes

• RemoteStoreMetadataRequest: Supports input parameters: index name and optional shard ID(s).
• RemoteStoreMetadataResponse: Contains per-shard metadata along with success/failure info
• RemoteStoreMetadata: Holds the parsed segment and translog metadata per shard.
• RemoteStoreMetadataAction: Registers the action under cluster:admin/remote_store/metadata.
• RemoteStoreMetadataRequestBuilder: Provides fluent client-side request building.

3. REST API: RestRemoteStoreMetadataAction

• New endpoints:
  • GET /_remotestore/metadata/{index}
  • GET /_remotestore/metadata/{index}/{shard_id} - Supports optional query param: ?local=true
• Internally calls the transport action and returns structured JSON.

4. Client Integration

• Updates to ClusterAdminClient and AbstractClient:
  • remoteStoreMetadata(RemoteStoreMetadataRequest, ActionListener)
  • prepareRemoteStoreMetadata(String index, String shardId)

Sample Response

GET "localhost:9200/_remotestore/metadata/my-index-1?pretty"

{
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "indices" : {
    "my-index-1" : {
      "shards" : {
        "0" : [
          {
            "index" : "my-index-1",
            "shard" : 0,
            "latest_segment_metadata_filename" : "metadata__9223372036854775806__9223372036854775804__9223372036854775803__9223372036854775805__-1084812520__9223370287071596565__2",
            "latest_translog_metadata_filename" : "metadata__9223372036854775806__9223372036854775804__9223370287071596713__-1084812520__9223372036854775804__1__1",
            "available_segment_metadata_files" : {
              "metadata__9223372036854775806__9223372036854775804__9223372036854775803__9223372036854775805__-1084812520__9223370287071596565__2" : {
                "files" : {
                  "_0.cfe" : {
                    "original_name" : "_0.cfe",
                    "checksum" : "624593751",
                    "length" : 517
                  },
                  "_0.si" : {
                    "original_name" : "_0.si",
                    "checksum" : "3210944240",
                    "length" : 324
                  },
                  "_0.cfs" : {
                    "original_name" : "_0.cfs",
                    "checksum" : "705023748",
                    "length" : 3933
                  },
                  "segments_3" : {
                    "original_name" : "segments_3",
                    "checksum" : "1916983394",
                    "length" : 230
                  }
                },
                "replication_checkpoint" : {
                  "segment_infos_version" : 9,
                  "codec" : "Lucene101",
                  "length" : 4774,
                  "created_timestamp" : 163477705159875,
                  "segments_gen" : 3,
                  "primary_term" : 1
                }
              },
              "metadata__9223372036854775806__9223372036854775804__9223372036854775804__9223372036854775806__-1084812520__9223370287071600820__2" : {
                "files" : {
                  "segments_3" : {
                    "original_name" : "segments_3",
                    "checksum" : "1916983394",
                    "length" : 230
                  }
                },
                "replication_checkpoint" : {
                  "segment_infos_version" : 7,
                  "codec" : "Lucene101",
                  "length" : 0,
                  "created_timestamp" : 163472814252458,
                  "segments_gen" : 3,
                  "primary_term" : 1
                }
              }
            },
            "available_translog_metadata_files" : {
              "metadata__9223372036854775806__9223372036854775804__9223370287071596713__-1084812520__9223372036854775804__1__1" : {
                "generation" : 3,
                "generation_to_primary_term" : {
                  "3" : "1"
                },
                "min_translog_gen" : 3,
                "primary_term" : 1
              },
              "metadata__9223372036854775806__9223372036854775805__9223370287071601448__-1084812520__9223372036854775805__1__1" : {
                "generation" : 2,
                "generation_to_primary_term" : {
                  "2" : "1"
                },
                "min_translog_gen" : 2,
                "primary_term" : 1
              }
            }
          }
        ]
      }
    }
  }
}

[github-actions]

❌ Gradle check result for 2e31b7f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for c3433c6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 73c9694: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for f706bd0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for f368d9e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for c1e02e2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for c013578: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 2a61841: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 432cc70: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for b3ed29d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 1476131: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for d92ef51: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 722d6dd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 722d6dd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for 722d6dd: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions]

❌ Gradle check result for 1d07935: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 1612b2f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?