Description
Describe the bug
Results are different for count_over_time()
when running an range or instant queries with different time boundaries
An example is the following, where the query is {}|count_over_time()with(exemplars=0)
and the time range is:
- for the range query a 10 min (
1748254200
..1748254800
) with steps of 1 min - for the instant query a 1 min range:
min | start | end | query_range (step=1m) | query (in 1 min steps) |
---|---|---|---|---|
0..1 | 1748254200 | 1748254260 | 11538 | 7102 |
1..2 | 1748254260 | 1748254320 | 12343 | 9403 |
2..3 | 1748254320 | 1748254380 | 13408 | 8722 |
3..4 | 1748254380 | 1748254440 | 11959 | 8314 |
4..5 | 1748254440 | 1748254500 | 11840 | 11327 |
5..6 | 1748254500 | 1748254560 | 13210 | 5145 |
6..7 | 1748254560 | 1748254620 | 12422 | 8473 |
7..8 | 1748254620 | 1748254680 | 12126 | 7296 |
8..9 | 1748254680 | 1748254740 | 12338 | 8419 |
9..10 | 1748254740 | 1748254800 | 12037 | 5294 |
sum | 123221 | 79495 |
Note 1: For the above the range query yields a different result for the whole range with a step of 10min
- query_range:
sum(count_over_time())
for 200..800 step=1m:123221
- query_range:
count_over_time()
for 200..800 step=10m:118635
- query:
count_over_time()
for 200..800:118635
Note 2: Other than this difference, the results are deterministic, where Tempo always returns the same results for the same input parameters.
Note 3: Extending the range query doesn't alter it's result for the overlapping range.
min | start | end | query_range (1748254200..1748254200) | query_range (1748254020..1748254980) |
---|---|---|---|---|
-3..-2 | 1748254020 | 1748254080 | - | 11942 |
-2..-1 | 1748254080 | 1748254140 | - | 13169 |
-1..0 | 1748254140 | 1748254200 | - | 12079 |
0..1 | 1748254200 | 1748254260 | 11538 | 11538 |
1..2 | 1748254260 | 1748254320 | 12343 | 12343 |
2..3 | 1748254320 | 1748254380 | 13408 | 13408 |
3..4 | 1748254380 | 1748254440 | 11959 | 11959 |
4..5 | 1748254440 | 1748254500 | 11840 | 11840 |
5..6 | 1748254500 | 1748254560 | 13210 | 13210 |
6..7 | 1748254560 | 1748254620 | 12422 | 12422 |
7..8 | 1748254620 | 1748254680 | 12126 | 12126 |
8..9 | 1748254680 | 1748254740 | 12338 | 12338 |
9..10 | 1748254740 | 1748254800 | 12037 | 12037 |
10..11 | 1748254800 | 1748254860 | - | 13218 |
11..12 | 1748254860 | 1748254920 | - | 12496 |
12..13 | 1748254920 | 1748254980 | - | 12364 |
Note 4: A range query and instant query that span the same time range yield the same results:
-
query_range:
count_over_time()
for 200..260 step=1m:7102
-
query:
count_over_time()
for 200..260:7102
-
query_range:
count_over_time()
for 200..800 step=10m:118635
-
query:
count_over_time()
for 200..800:118635
To Reproduce
Run a TraceQL metrics range query and instant query over the same time range with steps different than the time range, as described above.
Expected behavior
An instant query will produce the same result as a range query over the same time span.
Environment
- Infrastructure: Grafana Cloud
{
"version": "weekly-r198-ff5a82300",
"revision": "ff5a82300",
"branch": "weekly-r198",
"buildUser": "",
"buildDate": "",
"goVersion": "go1.24.1"
}