Skip to content

AWS Lambda Runtime legacy internal handlers need to be ignored from being instrumented and so traced … #10942

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

serkan-ozal
Copy link
Contributor

@serkan-ozal serkan-ozal commented Mar 23, 2024

... Otherwise, there might be double root spans.

Fixes #10931

In Java 8 and Java 11 runtimes, AWS Lambda runtime is packaged under lambdainternal package (it is com.amazonaws.services.lambda.runtime.api.client for new runtime likes Java 17 and Java 21)

I have

  • zipped /var/runtime/ folder in the AWS Lambda environment for Java 8 and Java 11 runtimes,
  • uploaded it to S3,
  • downloaded it to my local
  • and then unzipped

Here are the existing files:
Screenshot 2024-03-23 at 17 01 27

Then I have decompiled classes from LambdaSandboxJava-byol.jar and LambdaJavaRTEntry-byol.jar.

And as you can see below, AWS Lambda runtime internal classes are packaged under lambdainternal package and the internal handler (lambdainternal.EventHandlerLoader) we need to ignore is there.

Screenshot 2024-03-23 at 16 41 03

…eing instrumented and so traced. Otherwise, there might be double root spans.
Copy link
Member

@tylerbenson tylerbenson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow. Love the deep analysis you went through for this. That seems worthy of a blog post somewhere. Thanks for getting to the bottom of this.

Copy link
Member

@trask trask left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

…ain/java/io/opentelemetry/javaagent/instrumentation/awslambdacore/v1_0/AwsLambdaRequestHandlerInstrumentation.java

Co-authored-by: Trask Stalnaker <[email protected]>
@serkan-ozal
Copy link
Contributor Author

Hi @trask,

Thanks for the review, I have applied your suggestion.

@laurit laurit merged commit 9d94b4f into open-telemetry:main Mar 26, 2024
49 checks passed
srprash added a commit to aws-observability/aws-otel-java-instrumentation that referenced this pull request Feb 3, 2025
In the [previous
bugfix](#1000),
we missed porting this change from OTel specifically for Java 8 and 11
runtimes.

open-telemetry/opentelemetry-java-instrumentation#10942

**Testing:** Created an ADOT lambda layer with the change and verified
that the duplicate lambda spans do not appear for Java 11, 17, and 21.

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Double Server span create by lambda instrumentation
4 participants