webui: add server info to chat message #14065
Conversation
There was a problem hiding this comment.
Sorry this file needs to be removed
| <br />- Path: {msg.serverProps.model_path} | ||
| <br />- Build: {msg.serverProps.build_info} | ||
| <br />- Context: {msg.serverProps.n_ctx} |
There was a problem hiding this comment.
I did it this way to maintain homogeneity with the other dropdown. Let me if we should still use <ul> <li> here
<div className="dropdown-content bg-base-100 z-10 w-64 p-2 shadow mt-4">
<b>Prompt</b>
<br />- Tokens: {timings.prompt_n}
<br />- Time: {timings.prompt_ms} ms
<br />- Speed: {timings.prompt_per_second.toFixed(1)} t/s
<br />
<b>Generation</b>
<br />- Tokens: {timings.predicted_n}
<br />- Time: {timings.predicted_ms} ms
<br />- Speed: {timings.predicted_per_second.toFixed(1)} t/s
<br />
</div>
| className="cursor-pointer font-semibold text-sm opacity-60" | ||
| > | ||
| Model:{' '} | ||
| {msg.serverProps.model_path?.split(/(\\|\/)/).pop()} |
There was a problem hiding this comment.
this logic should be abstract into a function called modelNameFromPath()
| serverProps: serverProps | ||
| ? { | ||
| model_path: serverProps.model_path, | ||
| build_info: serverProps.build_info, | ||
| n_ctx: serverProps.n_ctx, | ||
| } | ||
| : null, |
There was a problem hiding this comment.
This is quite fragile for 2 reasons:
- If user changes model but forget to reload the frontend, this
serverPropsobject will be out of sync - This design is not future-proof. If we decide to add a new key into
serverPropsin the future, it will be quite messy. TheserverPropsobject is not supposed to be saved an reuse later
There was a problem hiding this comment.
@ngxson thanks for your review!
getServerProps is only called once on mount, so at the moment there is no way to get these in sync in case the backend changes but the frontend has not reloaded. One imprecise solution is to periodically poll the backend /health API and reload once the server comes back online. Let me know if that works
In case of future proofing the design, do you wish to de-couple these things? I think it might still be good to use the serverProps object to read these values but store them as separate types in the message.
Fixes #13570. Tested this change out locally, this a model card similar to the timing card, also added a similar checkbox to toggle this behavior.
Also checked if it has getting added to the IndexedDB
I purposefully did not store the whole
serverPropsobject because it was quite large