add documentation note about how to add instrumentation tests to CI [LANGPLAT-352] #9029

emmettbutler · 2025-06-24T18:25:34Z

What Does This Do

Adds a direct mention of how instrumentation tests get added to CI

Motivation

To make it easier for first-time contributors to become effective

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

github-actions · 2025-06-24T18:25:44Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Remove the tag from the pull request title

If you need help, please check our contributing guidelines.

pr-commenter · 2025-06-24T19:00:44Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	emmett.butler/ci-add-test-note
git_commit_date	1750788935	1750789457
git_commit_sha	`faeb62c`	`41fc85d`
release_version	1.51.0-SNAPSHOT~faeb62cfb1	1.51.0-SNAPSHOT~41fc85d783

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750791235	1750791235
ci_job_id	996644733	996644733
ci_pipeline_id	68648524	68648524
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zazkytx-project-304-concurrent-0-e1ixh0gd 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zazkytx-project-304-concurrent-0-e1ixh0gd 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 8 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~41fc85d783, baseline=1.51.0-SNAPSHOT~faeb62cfb1

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (995.138 ms) : 0, 995138
Total [baseline] (8.568 s) : 0, 8567876
Agent [candidate] (995.047 ms) : 0, 995047
Total [candidate] (8.562 s) : 0, 8562106
section iast
Agent [baseline] (1.137 s) : 0, 1136841
Total [baseline] (9.315 s) : 0, 9315482
Agent [candidate] (1.129 s) : 0, 1129137
Total [candidate] (9.307 s) : 0, 9306530

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	995.138 ms	-
Agent	iast	1.137 s	141.704 ms (14.2%)
Total	tracing	8.568 s	-
Total	iast	9.315 s	747.606 ms (8.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	995.047 ms	-
Agent	iast	1.129 s	134.091 ms (13.5%)
Total	tracing	8.562 s	-
Total	iast	9.307 s	744.424 ms (8.7%)

gantt
    title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~41fc85d783, baseline=1.51.0-SNAPSHOT~faeb62cfb1

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (686.694 ms) : 0, 686694
BytebuddyAgent [candidate] (686.297 ms) : 0, 686297
GlobalTracer [baseline] (242.661 ms) : 0, 242661
GlobalTracer [candidate] (242.919 ms) : 0, 242919
AppSec [baseline] (29.944 ms) : 0, 29944
AppSec [candidate] (29.953 ms) : 0, 29953
Debugger [baseline] (6.07 ms) : 0, 6070
Debugger [candidate] (6.064 ms) : 0, 6064
Remote Config [baseline] (648.7 µs) : 0, 649
Remote Config [candidate] (649.917 µs) : 0, 650
Telemetry [baseline] (8.17 ms) : 0, 8170
Telemetry [candidate] (8.198 ms) : 0, 8198
section iast
BytebuddyAgent [baseline] (813.233 ms) : 0, 813233
BytebuddyAgent [candidate] (807.15 ms) : 0, 807150
GlobalTracer [baseline] (233.519 ms) : 0, 233519
GlobalTracer [candidate] (232.168 ms) : 0, 232168
AppSec [baseline] (27.2 ms) : 0, 27200
AppSec [candidate] (28.995 ms) : 0, 28995
Debugger [baseline] (5.792 ms) : 0, 5792
Debugger [candidate] (5.757 ms) : 0, 5757
Remote Config [baseline] (573.596 µs) : 0, 574
Remote Config [candidate] (578.542 µs) : 0, 579
Telemetry [baseline] (7.826 ms) : 0, 7826
Telemetry [candidate] (7.813 ms) : 0, 7813
IAST [baseline] (27.768 ms) : 0, 27768
IAST [candidate] (25.769 ms) : 0, 25769

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~41fc85d783, baseline=1.51.0-SNAPSHOT~faeb62cfb1

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.007 s) : 0, 1006740
Total [baseline] (10.807 s) : 0, 10806955
Agent [candidate] (998.499 ms) : 0, 998499
Total [candidate] (10.769 s) : 0, 10768631
section appsec
Agent [baseline] (1.171 s) : 0, 1171398
Total [baseline] (10.795 s) : 0, 10794914
Agent [candidate] (1.185 s) : 0, 1184877
Total [candidate] (10.767 s) : 0, 10766696
section iast
Agent [baseline] (1.146 s) : 0, 1145807
Total [baseline] (10.888 s) : 0, 10887979
Agent [candidate] (1.129 s) : 0, 1128719
Total [candidate] (10.872 s) : 0, 10872180
section profiling
Agent [baseline] (1.241 s) : 0, 1241244
Total [baseline] (11.0 s) : 0, 11000132
Agent [candidate] (1.243 s) : 0, 1243321
Total [candidate] (11.103 s) : 0, 11102617

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.007 s	-
Agent	appsec	1.171 s	164.659 ms (16.4%)
Agent	iast	1.146 s	139.068 ms (13.8%)
Agent	profiling	1.241 s	234.504 ms (23.3%)
Total	tracing	10.807 s	-
Total	appsec	10.795 s	-12.041 ms (-0.1%)
Total	iast	10.888 s	81.025 ms (0.7%)
Total	profiling	11.0 s	193.177 ms (1.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	998.499 ms	-
Agent	appsec	1.185 s	186.378 ms (18.7%)
Agent	iast	1.129 s	130.22 ms (13.0%)
Agent	profiling	1.243 s	244.822 ms (24.5%)
Total	tracing	10.769 s	-
Total	appsec	10.767 s	-1.935 ms (-0.0%)
Total	iast	10.872 s	103.549 ms (1.0%)
Total	profiling	11.103 s	333.985 ms (3.1%)

gantt
    title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~41fc85d783, baseline=1.51.0-SNAPSHOT~faeb62cfb1

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (695.229 ms) : 0, 695229
BytebuddyAgent [candidate] (688.293 ms) : 0, 688293
GlobalTracer [baseline] (244.927 ms) : 0, 244927
GlobalTracer [candidate] (243.42 ms) : 0, 243420
AppSec [baseline] (30.416 ms) : 0, 30416
AppSec [candidate] (30.133 ms) : 0, 30133
Debugger [baseline] (6.124 ms) : 0, 6124
Debugger [candidate] (6.071 ms) : 0, 6071
Remote Config [baseline] (649.192 µs) : 0, 649
Remote Config [candidate] (645.883 µs) : 0, 646
Telemetry [baseline] (8.319 ms) : 0, 8319
Telemetry [candidate] (8.966 ms) : 0, 8966
section appsec
BytebuddyAgent [baseline] (708.941 ms) : 0, 708941
BytebuddyAgent [candidate] (719.019 ms) : 0, 719019
GlobalTracer [baseline] (236.03 ms) : 0, 236030
GlobalTracer [candidate] (237.65 ms) : 0, 237650
AppSec [baseline] (168.997 ms) : 0, 168997
AppSec [candidate] (170.39 ms) : 0, 170390
Debugger [baseline] (5.908 ms) : 0, 5908
Debugger [candidate] (5.907 ms) : 0, 5907
Remote Config [baseline] (604.841 µs) : 0, 605
Remote Config [candidate] (599.994 µs) : 0, 600
Telemetry [baseline] (8.155 ms) : 0, 8155
Telemetry [candidate] (8.168 ms) : 0, 8168
IAST [baseline] (21.937 ms) : 0, 21937
IAST [candidate] (22.063 ms) : 0, 22063
section iast
BytebuddyAgent [baseline] (819.419 ms) : 0, 819419
BytebuddyAgent [candidate] (806.134 ms) : 0, 806134
GlobalTracer [baseline] (234.952 ms) : 0, 234952
GlobalTracer [candidate] (232.283 ms) : 0, 232283
AppSec [baseline] (28.779 ms) : 0, 28779
AppSec [candidate] (29.857 ms) : 0, 29857
Debugger [baseline] (5.858 ms) : 0, 5858
Debugger [candidate] (5.779 ms) : 0, 5779
Remote Config [baseline] (586.194 µs) : 0, 586
Remote Config [candidate] (574.335 µs) : 0, 574
Telemetry [baseline] (7.999 ms) : 0, 7999
Telemetry [candidate] (7.852 ms) : 0, 7852
IAST [baseline] (27.145 ms) : 0, 27145
IAST [candidate] (25.323 ms) : 0, 25323
section profiling
ProfilingAgent [baseline] (102.136 ms) : 0, 102136
ProfilingAgent [candidate] (102.17 ms) : 0, 102170
BytebuddyAgent [baseline] (676.43 ms) : 0, 676430
BytebuddyAgent [candidate] (677.942 ms) : 0, 677942
GlobalTracer [baseline] (361.72 ms) : 0, 361720
GlobalTracer [candidate] (362.159 ms) : 0, 362159
AppSec [baseline] (32.602 ms) : 0, 32602
AppSec [candidate] (30.554 ms) : 0, 30554
Debugger [baseline] (8.972 ms) : 0, 8972
Debugger [candidate] (13.124 ms) : 0, 13124
Remote Config [baseline] (662.591 µs) : 0, 663
Remote Config [candidate] (658.061 µs) : 0, 658
Telemetry [baseline] (10.198 ms) : 0, 10198
Telemetry [candidate] (7.975 ms) : 0, 7975
Profiling [baseline] (102.162 ms) : 0, 102162
Profiling [candidate] (102.195 ms) : 0, 102195

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	emmett.butler/ci-add-test-note
git_commit_date	1750788935	1750789457
git_commit_sha	`faeb62c`	`41fc85d`
release_version	1.51.0-SNAPSHOT~faeb62cfb1	1.51.0-SNAPSHOT~41fc85d783

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750790905	1750790905
ci_job_id	996644734	996644734
ci_pipeline_id	68648524	68648524
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-a1vxyzsw-project-304-concurrent-0-nq5oezbi 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-a1vxyzsw-project-304-concurrent-0-nq5oezbi 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 3 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	worse [+272.616µs; +572.992µs] or [+3.128%; +6.574%]	unstable [-85.458op/s; +36.520op/s] or [-16.060%; +6.863%]	9.139ms	507.656op/s	8.716ms	532.125op/s
scenario:load:petclinic:no_agent:high_load	worse [+1.231ms; +1.878ms] or [+3.400%; +5.187%]	unstable [-14.323op/s; +3.773op/s] or [-11.096%; +2.923%]	37.766ms	123.812op/s	36.211ms	129.088op/s
scenario:load:petclinic:profiling:high_load	worse [+1.254ms; +2.304ms] or [+2.593%; +4.765%]	unstable [-10.577op/s; +3.777op/s] or [-10.930%; +3.903%]	50.137ms	93.375op/s	48.358ms	96.775op/s
scenario:load:petclinic:iast:high_load	better [-1.901ms; -1.064ms] or [-4.259%; -2.384%]	unstable [-4.363op/s; +11.463op/s] or [-4.161%; +10.932%]	43.146ms	108.412op/s	44.628ms	104.862op/s
scenario:load:petclinic:code_origins:high_load	better [-2.205ms; -1.412ms] or [-4.855%; -3.109%]	unstable [-3.336op/s; +11.936op/s] or [-3.238%; +11.584%]	43.598ms	107.338op/s	45.406ms	103.037op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~41fc85d783, baseline=1.51.0-SNAPSHOT~faeb62cfb1
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.298 ms) : 4249, 4347
.   : milestone, 4298,
iast (8.927 ms) : 8784, 9071
.   : milestone, 8927,
iast_FULL (14.012 ms) : 13730, 14293
.   : milestone, 14012,
iast_GLOBAL (9.775 ms) : 9600, 9950
.   : milestone, 9775,
profiling (8.716 ms) : 8580, 8852
.   : milestone, 8716,
tracing (7.655 ms) : 7540, 7769
.   : milestone, 7655,
section candidate
no_agent (4.396 ms) : 4347, 4445
.   : milestone, 4396,
iast (8.876 ms) : 8733, 9019
.   : milestone, 8876,
iast_FULL (13.796 ms) : 13518, 14075
.   : milestone, 13796,
iast_GLOBAL (9.913 ms) : 9739, 10087
.   : milestone, 9913,
profiling (9.139 ms) : 8996, 9282
.   : milestone, 9139,
tracing (7.836 ms) : 7721, 7950
.   : milestone, 7836,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.298 ms [4.249 ms, 4.347 ms]	-
iast	8.927 ms [8.784 ms, 9.071 ms]	4.629 ms (107.7%)
iast_FULL	14.012 ms [13.73 ms, 14.293 ms]	9.714 ms (226.0%)
iast_GLOBAL	9.775 ms [9.6 ms, 9.95 ms]	5.477 ms (127.4%)
profiling	8.716 ms [8.58 ms, 8.852 ms]	4.418 ms (102.8%)
tracing	7.655 ms [7.54 ms, 7.769 ms]	3.357 ms (78.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.396 ms [4.347 ms, 4.445 ms]	-
iast	8.876 ms [8.733 ms, 9.019 ms]	4.48 ms (101.9%)
iast_FULL	13.796 ms [13.518 ms, 14.075 ms]	9.4 ms (213.8%)
iast_GLOBAL	9.913 ms [9.739 ms, 10.087 ms]	5.517 ms (125.5%)
profiling	9.139 ms [8.996 ms, 9.282 ms]	4.743 ms (107.9%)
tracing	7.836 ms [7.721 ms, 7.95 ms]	3.44 ms (78.2%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~41fc85d783, baseline=1.51.0-SNAPSHOT~faeb62cfb1
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.211 ms) : 35929, 36494
.   : milestone, 36211,
appsec (47.437 ms) : 47018, 47857
.   : milestone, 47437,
code_origins (45.406 ms) : 45019, 45794
.   : milestone, 45406,
iast (44.628 ms) : 44239, 45018
.   : milestone, 44628,
profiling (48.358 ms) : 47908, 48807
.   : milestone, 48358,
tracing (44.609 ms) : 44224, 44994
.   : milestone, 44609,
section candidate
no_agent (37.766 ms) : 37448, 38084
.   : milestone, 37766,
appsec (47.564 ms) : 47141, 47987
.   : milestone, 47564,
code_origins (43.598 ms) : 43250, 43946
.   : milestone, 43598,
iast (43.146 ms) : 42758, 43534
.   : milestone, 43146,
profiling (50.137 ms) : 49613, 50661
.   : milestone, 50137,
tracing (43.782 ms) : 43428, 44137
.   : milestone, 43782,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.211 ms [35.929 ms, 36.494 ms]	-
appsec	47.437 ms [47.018 ms, 47.857 ms]	11.226 ms (31.0%)
code_origins	45.406 ms [45.019 ms, 45.794 ms]	9.195 ms (25.4%)
iast	44.628 ms [44.239 ms, 45.018 ms]	8.417 ms (23.2%)
profiling	48.358 ms [47.908 ms, 48.807 ms]	12.146 ms (33.5%)
tracing	44.609 ms [44.224 ms, 44.994 ms]	8.397 ms (23.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.766 ms [37.448 ms, 38.084 ms]	-
appsec	47.564 ms [47.141 ms, 47.987 ms]	9.798 ms (25.9%)
code_origins	43.598 ms [43.25 ms, 43.946 ms]	5.832 ms (15.4%)
iast	43.146 ms [42.758 ms, 43.534 ms]	5.38 ms (14.2%)
profiling	50.137 ms [49.613 ms, 50.661 ms]	12.371 ms (32.8%)
tracing	43.782 ms [43.428 ms, 44.137 ms]	6.016 ms (15.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	emmett.butler/ci-add-test-note
git_commit_date	1750788935	1750789457
git_commit_sha	`faeb62c`	`41fc85d`
release_version	1.51.0-SNAPSHOT~faeb62cfb1	1.51.0-SNAPSHOT~41fc85d783

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1750791507	1750791507
ci_job_id	996644735	996644735
ci_pipeline_id	68648524	68648524
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zazkytx-project-304-concurrent-1-5hfuupa9 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zazkytx-project-304-concurrent-1-5hfuupa9 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~41fc85d783, baseline=1.51.0-SNAPSHOT~faeb62cfb1
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.213 s) : 15213000, 15213000
.   : milestone, 15213000,
appsec (14.795 s) : 14795000, 14795000
.   : milestone, 14795000,
iast (18.448 s) : 18448000, 18448000
.   : milestone, 18448000,
iast_GLOBAL (17.963 s) : 17963000, 17963000
.   : milestone, 17963000,
profiling (15.35 s) : 15350000, 15350000
.   : milestone, 15350000,
tracing (15.114 s) : 15114000, 15114000
.   : milestone, 15114000,
section candidate
no_agent (15.462 s) : 15462000, 15462000
.   : milestone, 15462000,
appsec (14.819 s) : 14819000, 14819000
.   : milestone, 14819000,
iast (18.618 s) : 18618000, 18618000
.   : milestone, 18618000,
iast_GLOBAL (18.023 s) : 18023000, 18023000
.   : milestone, 18023000,
profiling (14.925 s) : 14925000, 14925000
.   : milestone, 14925000,
tracing (15.298 s) : 15298000, 15298000
.   : milestone, 15298000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.213 s [15.213 s, 15.213 s]	-
appsec	14.795 s [14.795 s, 14.795 s]	-418.0 ms (-2.7%)
iast	18.448 s [18.448 s, 18.448 s]	3.235 s (21.3%)
iast_GLOBAL	17.963 s [17.963 s, 17.963 s]	2.75 s (18.1%)
profiling	15.35 s [15.35 s, 15.35 s]	137.0 ms (0.9%)
tracing	15.114 s [15.114 s, 15.114 s]	-99.0 ms (-0.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.462 s [15.462 s, 15.462 s]	-
appsec	14.819 s [14.819 s, 14.819 s]	-643.0 ms (-4.2%)
iast	18.618 s [18.618 s, 18.618 s]	3.156 s (20.4%)
iast_GLOBAL	18.023 s [18.023 s, 18.023 s]	2.561 s (16.6%)
profiling	14.925 s [14.925 s, 14.925 s]	-537.0 ms (-3.5%)
tracing	15.298 s [15.298 s, 15.298 s]	-164.0 ms (-1.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~41fc85d783, baseline=1.51.0-SNAPSHOT~faeb62cfb1
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.478 ms) : 1466, 1489
.   : milestone, 1478,
appsec (2.411 ms) : 2362, 2460
.   : milestone, 2411,
iast (2.188 ms) : 2127, 2250
.   : milestone, 2188,
iast_GLOBAL (2.236 ms) : 2174, 2298
.   : milestone, 2236,
profiling (2.47 ms) : 2308, 2632
.   : milestone, 2470,
tracing (1.997 ms) : 1950, 2045
.   : milestone, 1997,
section candidate
no_agent (1.478 ms) : 1466, 1489
.   : milestone, 1478,
appsec (2.408 ms) : 2359, 2457
.   : milestone, 2408,
iast (2.195 ms) : 2134, 2256
.   : milestone, 2195,
iast_GLOBAL (2.231 ms) : 2169, 2293
.   : milestone, 2231,
profiling (2.024 ms) : 1975, 2073
.   : milestone, 2024,
tracing (2.011 ms) : 1963, 2058
.   : milestone, 2011,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.466 ms, 1.489 ms]	-
appsec	2.411 ms [2.362 ms, 2.46 ms]	932.794 µs (63.1%)
iast	2.188 ms [2.127 ms, 2.25 ms]	710.422 µs (48.1%)
iast_GLOBAL	2.236 ms [2.174 ms, 2.298 ms]	757.994 µs (51.3%)
profiling	2.47 ms [2.308 ms, 2.632 ms]	992.091 µs (67.1%)
tracing	1.997 ms [1.95 ms, 2.045 ms]	519.616 µs (35.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.466 ms, 1.489 ms]	-
appsec	2.408 ms [2.359 ms, 2.457 ms]	929.875 µs (62.9%)
iast	2.195 ms [2.134 ms, 2.256 ms]	716.996 µs (48.5%)
iast_GLOBAL	2.231 ms [2.169 ms, 2.293 ms]	753.218 µs (51.0%)
profiling	2.024 ms [1.975 ms, 2.073 ms]	545.819 µs (36.9%)
tracing	2.011 ms [1.963 ms, 2.058 ms]	532.798 µs (36.1%)

amarziali · 2025-06-26T07:10:11Z

docs/how_instrumentations_work.md

@@ -677,6 +677,8 @@ Tests are written in Groovy using the [Spock framework](http://spockframework.or
 instrumentations, `AgentTestRunner` must be extended. For example, HTTP server frameworks use base tests which enforce
 consistency between different implementations (
 see [HttpServerTest](../dd-java-agent/testing/src/main/groovy/datadog/trace/agent/test/base/HttpServerTest.groovy)).
+Such Groovy-based tests are included by default in the CI instrumentation test jobs.


I would have said that test marked as @Flaky are not executed by the CI

add documentation note about how to add instrumentation tests to CI

41fc85d

emmettbutler requested a review from a team as a code owner June 24, 2025 18:25

emmettbutler requested a review from amarziali June 24, 2025 18:25

emmettbutler requested review from andrewlock and removed request for andrewlock June 24, 2025 18:25

amarziali added comp: testing Testing type: documentation tag: no release notes Changes to exclude from release notes labels Jun 26, 2025

amarziali reviewed Jun 26, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add documentation note about how to add instrumentation tests to CI [LANGPLAT-352] #9029

add documentation note about how to add instrumentation tests to CI [LANGPLAT-352] #9029

Uh oh!

emmettbutler commented Jun 24, 2025

Uh oh!

github-actions bot commented Jun 24, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Jun 24, 2025

Uh oh!

amarziali Jun 26, 2025

Uh oh!

Uh oh!

add documentation note about how to add instrumentation tests to CI [LANGPLAT-352] #9029

Are you sure you want to change the base?

add documentation note about how to add instrumentation tests to CI [LANGPLAT-352] #9029

Uh oh!

Conversation

emmettbutler commented Jun 24, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions bot commented Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Jun 24, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

amarziali Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Jun 24, 2025 •

edited

Loading