Skip to content

[easy] Pass compile_fx only the config patches #16845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 20, 2025
Merged

[easy] Pass compile_fx only the config patches #16845

merged 1 commit into from
Apr 20, 2025

Conversation

zou3519
Copy link
Collaborator

@zou3519 zou3519 commented Apr 18, 2025 •
edited by github-actions bot
Loading

Previously we were passing compile_fx the entire default inductor config with the patches applied to it.

compile_fx only needs to be passed the patches (reference: https://github.com/pytorch/pytorch/blob/29317f8585ecb232412df3f39734490f0f6d8230/torch/_inductor/compile_fx.py#L1873-L1880)

This PR changes vLLM to only pass the patches. This makes debugging things easier (I can stare at just the delta and see what vLLM changed).

Test Plan:
I ran the following command and verified that performance didn't change.

VLLM_USE_V1=1 python benchmark_latency.py --model meta-llama/Meta-Llama-3-8B --batch-size 1 -O '{"level": 3, "compile_sizes": {1, 2}}'

@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@zou3519 zou3519 marked this pull request as ready for review April 18, 2025 15:05
tlrmchlsmth
tlrmchlsmth approved these changes Apr 18, 2025
View reviewed changes
Copy link
Collaborator

@tlrmchlsmth tlrmchlsmth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, lgtm

@tlrmchlsmth tlrmchlsmth added the ready ONLY add when PR is ready to merge/full CI is needed label Apr 18, 2025
@tlrmchlsmth tlrmchlsmth enabled auto-merge (squash) April 18, 2025 15:27
mgoin
mgoin approved these changes Apr 18, 2025
View reviewed changes
Copy link
Member

@mgoin mgoin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@DarkLight1337
Copy link
Member

Can you merge from main to fix CI?

youkaichao
youkaichao approved these changes Apr 19, 2025
View reviewed changes
Copy link
Member

@youkaichao youkaichao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zou3519
[easy] Pass compile_fx only the config patches
846ad03 
Previously we were passing compile_fx the entire default inductor config
with the patches applied to it.

compile_fx only needs to be passed the patches (reference:
https://github.com/pytorch/pytorch/blob/29317f8585ecb232412df3f39734490f0f6d8230/torch/_inductor/compile_fx.py#L1873-L1880)

This PR changes vLLM to only pass the patches. This makes debugging
things easier (I can stare at just the delta and see what vLLM changed).

Test Plan:
I ran the following command and verified that performance didn't change.
```
VLLM_USE_V1=1 python benchmark_latency.py --model meta-llama/Meta-Llama-3-8B --batch-size 1 -O '{"level": 3, "compile_sizes": {1, 2}}'
```

Signed-off-by: rzou <[email protected]>
auto-merge was automatically disabled April 19, 2025 14:36

Head branch was pushed to by a user without write access

@DarkLight1337 DarkLight1337 merged commit fe742ae into vllm-project:main Apr 20, 2025
43 checks passed
@github-project-automation github-project-automation bot moved this from To triage to Done in torch.compile integration Apr 20, 2025
yangw-dev pushed a commit to yangw-dev/vllm that referenced this pull request Apr 21, 2025
@zou3519 @yangw-dev
[easy] Pass compile_fx only the config patches (vllm-project#16845)
bf90580 
Signed-off-by: rzou <[email protected]>
Signed-off-by: Yang Wang <[email protected]>
dtransposed pushed a commit to dtransposed/vllm that referenced this pull request Apr 22, 2025
@zou3519
[easy] Pass compile_fx only the config patches (vllm-project#16845)
a8f1432 
Signed-off-by: rzou <[email protected]>
Signed-off-by: dtransposed <damian@damian-ml-machine.europe-west3-b.c.jetbrains-grazie.internal>
liuzijing2014 pushed a commit to liuzijing2014/vllm that referenced this pull request Apr 25, 2025
@zou3519 @liuzijing2014
[easy] Pass compile_fx only the config patches (vllm-project#16845)
e057906 
Signed-off-by: rzou <[email protected]>
Signed-off-by: Zijing Liu <[email protected]>
liuzijing2014 pushed a commit to liuzijing2014/vllm that referenced this pull request Apr 25, 2025
@zou3519 @liuzijing2014
[easy] Pass compile_fx only the config patches (vllm-project#16845)
a062d24 
Signed-off-by: rzou <[email protected]>
wuisawesome pushed a commit to character-tech/vllm that referenced this pull request Apr 28, 2025
@zou3519 @wuisawesome
[easy] Pass compile_fx only the config patches (vllm-project#16845)
a054cec 
Signed-off-by: rzou <[email protected]>
jikunshang pushed a commit to jikunshang/vllm that referenced this pull request Apr 29, 2025
@zou3519 @dbyoung18
[easy] Pass compile_fx only the config patches (vllm-project#16845)
c73af87 
Signed-off-by: rzou <[email protected]>
lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Apr 29, 2025
@zou3519 @lk-chen
[easy] Pass compile_fx only the config patches (vllm-project#16845)
5aa1069 
Signed-off-by: rzou <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tlrmchlsmth tlrmchlsmth tlrmchlsmth approved these changes

@mgoin mgoin mgoin approved these changes

@youkaichao youkaichao youkaichao approved these changes

@houseroad houseroad houseroad approved these changes

Assignees
No one assigned
Labels
ready ONLY add when PR is ready to merge/full CI is needed torch.compile
Projects
torch.compile integration
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@zou3519 @DarkLight1337 @tlrmchlsmth @mgoin @youkaichao @houseroad