-
Notifications
You must be signed in to change notification settings - Fork 3.1k
fix: reverted the URL of llama.cpp back to 'completion'. #5726
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
✅ Deploy Preview for continuedev canceled.
|
All contributors have signed the CLA ✍️ ✅ |
I have read the CLA Document and I hereby sign the CLA |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
For some extra background, I think we may actually want to remove this provider in the future, given that llama.cpp is now OpenAI-compatible. That means the OpenAI provider can be used instead.
Description
Closes: #5530
Reverted the URL of llama.cpp back to 'completion' since it's not changed at all.
The latest documentation from llama.cpp:
https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md#post-completion-given-a-prompt-it-returns-the-predicted-completion
/v1/completions
is for OAI-compatible clients only, but in LlamaCpp.ts there is no/v1
presented, and I don't think it should.Tests
Local tests with the extension built.