Skip to content

Offline symbol resolving improvements (parallel image resolving, new fast libdw based resolving backend) #1026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

tiago-rodrigues
Copy link

@tiago-rodrigues tiago-rodrigues commented Apr 14, 2025

  • Add parallel implementation of offline symbol resolving (process images in parallel, use '-m' to specify the level of parallelism, -1 [default] is max core count)
  • Add a couple of more offline symbol resolving backend implementations for linux (use '-n' to choose the backend to use):
    • a libdw based backend that is ~100x faster than addr2line and LGPL licensed (requires libdw package to be installed and LIBDW_OFFLINE_SYMBOL_RESOLUTION_SUPPORT to be enabled)
    • a libacktrace based backend (using the libbacktrace version that exists in the tracy codebase). This backend is not fully functional as it need to load the .so into memory to resolve the symbols which is not always possible (enable with LIBBACKTRACE_OFFLINE_SYMBOL_RESOLUTION_SUPPORT)
  • Add option to exclude certain images from being resolved (specified via -x)

…images in parallel)

- Add a few  different offline symbol resolving backend implementations for linux (libdw being the default, fastest and LGPL)
- Add option to exclude certain images from being resolved
@tiago-rodrigues
Copy link
Author

tiago-rodrigues commented Apr 14, 2025

@Lectem : this is the PR I was mentioning here: #1024 (comment)

@wolfpld : what would be the right way to add the dependency to "libdw"? As it's significantly faster than addr2line, I would like to make it the default offline resolver in the update util.
Would the the only way be to use CPM and clone and build elfutils/the libdw part of it at least?

I left the backtrace implementation for reference as @Lectem mentioned he wanted to attempt implementing it, I however never succeeded in getting it to work for all .so's maybe there is a way. Still I think libdw is the best option at the cost of this extra dependency.

@tiago-rodrigues tiago-rodrigues changed the title Offline symbol resolving improvements (paralell image resolving, new fast libdw based resolving backend) Offline symbol resolving improvements (parallel image resolving, new fast libdw based resolving backend) Apr 14, 2025
@Lectem
Copy link
Collaborator

Lectem commented Apr 14, 2025

Interesting, so from what I saw, we could use the offline_callbacks to check that the build-id is correct ? One of the main issues I'd like to fix is the that we might not be using the correct debug information. (Both when using update or with the server once we finish working on #1009 ).
On Windows this can be done by providing the PDB GUID, and on linux using the build-id.

If we do integrate libDw and that it supports build-id checks and we can do file lookup with debuginfod, then there'd be less/little need for modifying libbacktrace. (Except that libbacktrace supports more formats, could be used on platforms other than Linux, and doesn't require installing a package).

@wolfpld
Copy link
Owner

wolfpld commented Apr 14, 2025

If we do integrate libDw and that it supports build-id checks and we can do file lookup with debuginfod, then there'd be less/little need for modifying libbacktrace. (Except that libbacktrace supports more formats, could be used on platforms other than Linux, and doesn't require installing a package).

There are changes in Tracy libbacktrace that are required for inline functions to be properly reported.

@tiago-rodrigues
Copy link
Author

I've made the new back-ends optional so it passes on CI and still allows users to enable them if they want. Although I would like the libdw backend to the the default, unfortunately I don't see how I would do that with CPM and pulling and building elfutils to do this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants