Skip to content

Support time partitioned store gateway #5585

Open
@yeya24

Description

@yeya24

Is your feature request related to a problem? Please describe.
Thanos supports something called time based partitioning https://thanos.io/tip/components/store.md/#time-based-partitioning on Store Gateway. This is a good way to do hot/cold data tiering as relatively recent data (within 1 month) in store gateway is accessed much more often than older data (older than 1 month).

Time partitioned store gateways can configure different replication factors and other configurations, which helps to reduce SG memory consumption. (Can use a larger replication factor for hot data and smaller RF for cold data).

Describe the solution you'd like
There are multiple ways to do this but both requires some non trivial code changes.

  1. Querier supports multiple Store Gateway Ring and routes requests accordingly
  2. Querier still uses one SG ring, but X% of SGs in the ring are hot SGs and other SGs are cold SGs. Tiered SGs are basically two subrings and blocks will be assigned to the subrings based on the time range and block ID hash.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions