mirror of https://github.com/hwchase17/langchain
Fix wrong token counts from `get_num_tokens` from openai llms (#2952)
The encoding fetch was out of date. Luckily OpenAI has a nice[
`encoding_for_model`](46287bfa49/tiktoken/model.py
)
function in `tiktoken` we can use now.
pull/2960/head
parent
fea5619ce9
commit
b9db20481f
Loading…
Reference in New Issue