Probability / logit of output tokens #2790
Unanswered
Simratpreet
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I am trying to log the probability/logit of output token in top K sampling kernel. This is the place where I am trying to log.
My understanding is that outputLogProbs should contain the probability but it is always 0. Also my understanding is
sSum
variable should contain sum of all logits, but it is always equal to expLogit.here is actual output for one of the selected tokens:
outputLogProbs[curSeqLen * maxBatchSize + batchSlot]= 0.000000
sSum: 0.000201, logSum: -8.511453, logprob: -8.511453, explogit: 0.000201
In summary:
Beta Was this translation helpful? Give feedback.
All reactions