Skip to content

chore(iast): reduce telemetry error messages #13600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Jun 5, 2025

Conversation

avara1986
Copy link
Member

@avara1986 avara1986 commented Jun 5, 2025

This PR changes how IAST reports errors when an exception occurs, especially in cases where execution happens outside of context. This was originally implemented to analyze which services were triggering this issue, and we used that insight to refactor how IAST creates and destroys context.

However, after that refactor, this information no longer provides value and only adds noise. So now, those telemetry metrics are only emitted if IAST is in debug mode.

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

@avara1986 avara1986 added changelog/no-changelog A changelog entry is not required for this PR. ASM Application Security Monitoring labels Jun 5, 2025
Copy link
Contributor

github-actions bot commented Jun 5, 2025

CODEOWNERS have been resolved as:

ddtrace/appsec/_iast/_langchain.py                                      @DataDog/asm-python
ddtrace/appsec/_iast/_logs.py                                           @DataDog/asm-python
ddtrace/appsec/_iast/_metrics.py                                        @DataDog/asm-python
ddtrace/appsec/_iast/_taint_tracking/taint_tracking/taint_range.cpp     @DataDog/asm-python
tests/appsec/iast/aspects/test_fstrings.py                              @DataDog/asm-python
tests/appsec/iast/test_overhead_control_engine.py                       @DataDog/asm-python
tests/appsec/iast/test_telemetry.py                                     @DataDog/asm-python

Copy link
Contributor

github-actions bot commented Jun 5, 2025

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 272 ± 2 ms.

The average import time from base is: 273 ± 2 ms.

The import time difference between this PR and base is: -0.75 ± 0.09 ms.

Import time breakdown

The following import paths have shrunk:

ddtrace.auto 1.843 ms (0.68%)
ddtrace.bootstrap.sitecustomize 1.180 ms (0.43%)
ddtrace.bootstrap.preload 1.180 ms (0.43%)
ddtrace.internal.remoteconfig.client 0.629 ms (0.23%)
ddtrace 0.664 ms (0.24%)
ddtrace.internal._unpatched 0.021 ms (0.01%)

@pr-commenter
Copy link

pr-commenter bot commented Jun 5, 2025

Benchmarks

Benchmark execution time: 2025-06-05 14:17:33

Comparing candidate commit e8ec2d9 in PR branch avara1986/APPSEC-57578-iast_telemetry_logs with baseline commit 0e5abc9 in branch main.

Found 4 performance improvements and 2 performance regressions! Performance is the same for 551 metrics, 7 unstable metrics.

scenario:iastaspects-ljust_aspect

  • 🟩 execution_time [-2.716µs; -2.568µs] or [-20.877%; -19.737%]

scenario:iastaspects-lstrip_aspect

  • 🟩 execution_time [-2.551µs; -2.408µs] or [-19.685%; -18.579%]

scenario:iastaspects-rstrip_aspect

  • 🟩 execution_time [-2.566µs; -2.413µs] or [-19.946%; -18.759%]

scenario:iastaspects-strip_aspect

  • 🟩 execution_time [-2.623µs; -2.542µs] or [-20.165%; -19.543%]

scenario:iastaspectsospath-ospathbasename_aspect

  • 🟥 execution_time [+653.359ns; +800.890ns] or [+15.446%; +18.934%]

scenario:iastaspectsospath-ospathsplitdrive_aspect

  • 🟥 execution_time [+401.969ns; +454.032ns] or [+11.104%; +12.542%]

@avara1986 avara1986 marked this pull request as ready for review June 5, 2025 11:31
@avara1986 avara1986 requested a review from a team as a code owner June 5, 2025 11:31
avara1986 added a commit that referenced this pull request Jun 5, 2025
Fix IAST validation in fstring aspect. We're considering as unexpected
error, format errors like: `Unknown format code 'd' for object of type
'str'`

this PR fixes an error in
https://github.com/DataDog/dd-trace-py/pull/13600/files#r2128401733

part of #13600

APPSEC-57578

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
@avara1986 avara1986 marked this pull request as draft June 5, 2025 12:51
@avara1986 avara1986 marked this pull request as ready for review June 5, 2025 16:30
@avara1986 avara1986 enabled auto-merge (squash) June 5, 2025 16:33
@avara1986 avara1986 merged commit ce519c3 into main Jun 5, 2025
509 checks passed
@avara1986 avara1986 deleted the avara1986/APPSEC-57578-iast_telemetry_logs branch June 5, 2025 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ASM Application Security Monitoring changelog/no-changelog A changelog entry is not required for this PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants