-
Notifications
You must be signed in to change notification settings - Fork 656
feat(metrics): add a relation info panel in grafana #22196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds a new gauge metric, relation_info, to expose relation (table, source, sink) information in Grafana. The changes include introducing the new metric in the MetaMetrics struct with its refresh logic, extending the catalog controller with new methods to retrieve object information, and updating the Grafana dashboard to display the relation info panel.
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
File | Description |
---|---|
src/meta/src/rpc/metrics.rs | Added relation_info gauge metric and its refresh function. |
src/meta/src/controller/catalog/get_op.rs | Added methods to list table, source, and sink objects. |
src/meta/model/src/object.rs | Introduced a new Schema2 relation mapping annotation. |
grafana/dashboard/dev/actor_info.py | Extended dashboard with a panel to show relation info metrics. |
src/meta/src/rpc/metrics.rs
Outdated
@@ -193,6 +193,8 @@ pub struct MetaMetrics { | |||
pub table_info: IntGaugeVec, | |||
/// A dummy gauge metrics with its label to be the mapping from actor id to sink id | |||
pub sink_info: IntGaugeVec, | |||
/// A dummpy gauge metrics with its label to be relation info |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct the spelling error in the comment: replace 'dummpy' with 'dummy'.
/// A dummpy gauge metrics with its label to be relation info | |
/// A dummy gauge metrics with its label to be relation info |
Copilot uses AI. Check for mistakes.
// Output: Vec<(table id, db name, schema name, table name, resource group)> | ||
pub async fn list_table_objects( | ||
&self, | ||
) -> MetaResult<Vec<(TableId, String, String, String, String)>> { | ||
let inner = self.inner.read().await; | ||
Ok(Object::find() | ||
.select_only() | ||
.join(JoinType::InnerJoin, object::Relation::Table.def()) | ||
.join(JoinType::InnerJoin, object::Relation::Database2.def()) | ||
.join(JoinType::InnerJoin, object::Relation::Schema2.def()) | ||
.column(object::Column::Oid) | ||
.column(database::Column::Name) | ||
.column(schema::Column::Name) | ||
.column(table::Column::Name) | ||
.column(database::Column::ResourceGroup) | ||
.into_tuple() | ||
.all(&inner.db) | ||
.await?) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] The implementations of list_table_objects, list_source_objects, and list_sink_objects share similar query logic. Consider refactoring the common parts into a helper function to improve maintainability.
// Output: Vec<(table id, db name, schema name, table name, resource group)> | |
pub async fn list_table_objects( | |
&self, | |
) -> MetaResult<Vec<(TableId, String, String, String, String)>> { | |
let inner = self.inner.read().await; | |
Ok(Object::find() | |
.select_only() | |
.join(JoinType::InnerJoin, object::Relation::Table.def()) | |
.join(JoinType::InnerJoin, object::Relation::Database2.def()) | |
.join(JoinType::InnerJoin, object::Relation::Schema2.def()) | |
.column(object::Column::Oid) | |
.column(database::Column::Name) | |
.column(schema::Column::Name) | |
.column(table::Column::Name) | |
.column(database::Column::ResourceGroup) | |
.into_tuple() | |
.all(&inner.db) | |
.await?) | |
// Helper function to list objects with customizable columns and relations | |
async fn list_objects<T>( | |
&self, | |
columns: Vec<impl ColumnTrait>, | |
relations: Vec<impl RelationTrait>, | |
) -> MetaResult<Vec<T>> { | |
let inner = self.inner.read().await; | |
let mut query = Object::find().select_only(); | |
for relation in relations { | |
query = query.join(JoinType::InnerJoin, relation.def()); | |
} | |
for column in columns { | |
query = query.column(column); | |
} | |
Ok(query.into_tuple().all(&inner.db).await?) | |
} | |
// Output: Vec<(table id, db name, schema name, table name, resource group)> | |
pub async fn list_table_objects( | |
&self, | |
) -> MetaResult<Vec<(TableId, String, String, String, String)>> { | |
self.list_objects( | |
vec![ | |
object::Column::Oid, | |
database::Column::Name, | |
schema::Column::Name, | |
table::Column::Name, | |
database::Column::ResourceGroup, | |
], | |
vec![ | |
object::Relation::Table, | |
object::Relation::Database2, | |
object::Relation::Schema2, | |
], | |
) | |
.await |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
e6d5ee0
to
a2e51fd
Compare
a2e51fd
to
d9cff8b
Compare
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Resolve #22156. I try exposing the db and schema label in the specific metrics like source/sink throughput first but it turns out too difficult to do so. Given that the labels in tsdb won't be store repeatedly, the current approach is a good workaround IMO.
Checklist
Documentation
Release note