Skip to content

example/speculative: drafting fails completely when params.sparams.temp is set to 0 #5657

Closed
@mscheong01

Description

@mscheong01

Please include information about your system, the steps to reproduce the bug, and the version of llama.cpp that you are using. If possible, please provide a minimal code example that reproduces the bug.

In the current speculative.cpp implementation, params.sparams.temp is forced to -1.0f
However, if I change this value to 0:
image

draft sampling seems to fail completely:
image
(speculative.log)

Is this intended behavior?
I'm working on #5625 which removes the temperature limit so I'd like to get this fixed

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions