mirror of https://github.com/nomic-ai/gpt4all
sampling: remove incorrect offset for n_vocab (#900)
no effect, but avoids a *potential* bug later if we use actualVocabSize - which is for when a model has a larger embedding tensor/# of output logits than actually trained token to allow room for adding extras in finetuning - presently all of our models have had "placeholder" tokens in the vocab so this hasn't broken anything, but if the sizes did differ we want the equivalent of `logits[actualVocabSize:]` (the start point is unchanged), not `logits[-actualVocabSize:]` (this.)pull/913/head
parent
c6249c5664
commit
6624d7b2dd
Loading…
Reference in New Issue