Skip to content

feat: Cache sin cos in model instead of global LRU cache. #3378

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Apr 14, 2025
Merged

feat: Cache sin cos in model instead of global LRU cache. #3378

merged 3 commits into from
Apr 14, 2025

Conversation

yuxianq
Copy link
Collaborator

@yuxianq yuxianq commented Apr 8, 2025

No description provided.

@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 8, 2025

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #1496 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #1496 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1115 completed with status: 'FAILURE'

@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 9, 2025

/bot run --disable-fail-fast

@yuxianq yuxianq requested review from hlu1, kaiyux and QiJune April 9, 2025 02:33
@tensorrt-cicd
Copy link
Collaborator

PR_Github #1527 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #1527 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1141 completed with status: 'FAILURE'

@yuxianq yuxianq force-pushed the model-level-cache branch from 30f7fb9 to 105c73b Compare April 10, 2025 08:40
@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 10, 2025

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #1746 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #1746 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1298 completed with status: 'FAILURE'

@yuxianq yuxianq requested a review from litaotju April 11, 2025 07:38
@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 11, 2025

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #1896 [ run ] triggered by Bot

@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 11, 2025

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #1943 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #1896 [ run ] completed with state ABORTED

@tensorrt-cicd
Copy link
Collaborator

PR_Github #1943 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1430 completed with status: 'FAILURE'

@yuxianq yuxianq force-pushed the model-level-cache branch from 5ce7c3a to b7f47ed Compare April 12, 2025 06:08
@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 12, 2025

This PR is blocked by #3449 since the multi-gpu tests cannot pass without it.

@yuxianq yuxianq force-pushed the model-level-cache branch from b7f47ed to e0f4c56 Compare April 12, 2025 09:45
@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 12, 2025

/bot run --disable-fail-fast --stage-list "DGX_H100-4_GPUs-TensorRT-2"

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2013 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2013 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1482 (Partly Tested) completed with status: 'SUCCESS'

@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 12, 2025

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2015 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2015 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1484 completed with status: 'SUCCESS'

@yuxianq yuxianq enabled auto-merge (squash) April 14, 2025 02:40
@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 14, 2025

/bot reuse-pipeline

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2093 [ reuse-pipeline ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2093 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #2015 for commit 9154d24

@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 14, 2025

/bot reuse-pipeline

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2098 [ reuse-pipeline ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2098 [ reuse-pipeline ] completed with state FAILURE
Can't reuse PR_Github #0 with status: UNKNOWN

@yuxianq
Copy link
Collaborator Author

yuxianq commented Apr 14, 2025

/bot reuse-pipeline

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2102 [ reuse-pipeline ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2102 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #2015 for commit c0f8822

@yuxianq yuxianq merged commit 9d64b6b into NVIDIA:main Apr 14, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hlu1 hlu1 hlu1 left review comments

@QiJune QiJune QiJune left review comments

@kaiyux kaiyux kaiyux approved these changes

@litaotju litaotju Awaiting requested review from litaotju

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@yuxianq @tensorrt-cicd @hlu1 @QiJune @kaiyux