mirror of https://github.com/hwchase17/langchain
Update LlamaCpp parameters (#2411)
Add `n_batch` and `last_n_tokens_size` parameters to the LlamaCpp class. These parameters (epecially `n_batch`) significantly effect performance. There's also a `verbose` flag that prints system timings on the `Llama` class but I wasn't sure where to add this as it conflicts with (should be pulled from?) the LLM base class.pull/2419/head
parent
b026a62bc4
commit
e519a81a05
Loading…
Reference in New Issue