Feature Request: Top n-sigma sampler

[Ph0rk0z]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

It's another good sampler like XTC and DRY. Was just added recently: ggml-org/llama.cpp#13264

I've not checked to see how different sampling is here from mainline and if it's possible to just copy the PR or if that is a nono.

Motivation

I see people using/recommending it and do not have it :P

Seems like relatively low hanging fruit on the surface, unlike, say vision in the server. (where we don't have a good large MoE with vision; llama doesn't count)

Possible Implementation

No response

[ikawrakow]

So, the quoted PR just integrates it into the standard llama.cpp sampling mechanism. The actual sampler is implemented in their PR 11233. I looked at 11233, and it is a pretty trivial thing, so very easy to implement. I had never actually looked at the sampling code, but a quick check shows that it is not a copy/paste. Also this has been completely reorganized in mainline (they just love pushing pieces of code from here to there). Here sampling is part of common, over there it is now part of llama.cpp itself. So, adding a new sampler involves me first getting familiar with how sampling is done in this fork.

[Ph0rk0z]

#489