feat: add support for vLLM response format in reranking logic and up… #5954
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Your cubic subscription is currently inactive. Please reactivate your subscription to receive AI reviews and use cubic. |
❌ Deploy Preview for continuedev failed. Why did it fail? →
|
dosubot
bot
added
the
size:M
Iamshankhadeep
changed the title
sestinj
requested changes
Jun 4, 2025
There was a problem hiding this comment.
@Iamshankhadeep thanks for looking into this! I'm thinking that instead of adding this extra logic to the BaseLLM class, we could just add a rerank method to the dedicated VLLM class. You can find an example of such a rerank method in the Voyage class.
Let me know if this makes sense or if I can explain further
|
@sestinj yup totally makes sense, |
Iamshankhadeep
force-pushed
the
fix/fts-query-bug
branch
from
2f655fa to
c8d9d95
Compare
|
@sestinj should be ready for review. Let me know which issue I can work on next. thanks 🔥 |
|
✨ No issues found! Your code is sparkling clean! ✨ |
sestinj
approved these changes
Jun 16, 2025
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
vLLM's rerank endpoint returns results.results instead of results.data like everyone else. Our code was trying to sort undefined.
her's the link Re-rank API
I aslo added proper TypeScript types (VllmRerankResponse), but build is failing because of that.
Checklist
fixes #5869