Skip to content

[Feature] Support custom TopN pre-aggregation rules #13194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
2 of 3 tasks
wu-sheng opened this issue Apr 22, 2025 · 1 comment
Open
2 of 3 tasks

[Feature] Support custom TopN pre-aggregation rules #13194

wu-sheng opened this issue Apr 22, 2025 · 1 comment
Assignees
Labels
backend OAP backend related. database BanyanDB - SkyWalking native database feature New feature

Comments

@wu-sheng
Copy link
Member

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

Currently, we have pre-aggregation topN endpoint on the service page. Meanwhile, in the root page(e.g. layer=general), we have a layer-level topN based on attr0.
Right now, we can't do pre-aggregation as we have limited the topN rules defined in source(OAL) through @ScopeDefaultColumn.BanyanDB(groupByCondInTopN = true), which is hard-coded and only supports one rule.

Use case

In the current scenario, to speed up these two kinds(maybe in the future), we should support more rules in the bydb.yml for customization requirements. Different users could have different scale data, which sometimes requires more preaggregation to achieve CPU-speed-disk_volume tradeoff -> Less CPU cost, and faster query performance, to trade off more disk_volume as pre-aggregation.

Related issues

No response

Are you willing to submit a pull request to implement this on your own?

  • Yes I am willing to submit a pull request on my own!

Code of Conduct

@wu-sheng wu-sheng added backend OAP backend related. database BanyanDB - SkyWalking native database feature New feature labels Apr 22, 2025
@wu-sheng
Copy link
Member Author

@ScopeDefaultColumn.BanyanDB(groupByCondInTopN = true) and BanyanDB.TopNAggregation(in generated codes) could be considered to be removed, and we could only add those topN from the file configurations. Of course, we need some naming mechanism for query.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend OAP backend related. database BanyanDB - SkyWalking native database feature New feature
Projects
None yet
Development

No branches or pull requests

2 participants