Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-11-10 01:10:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(community): allow support for disabling max_tokens args (#21534)

Browse Source 
This PR fixes an issue with not able to use unlimited/infinity tokens
from the respective provider for the LiteLLM provider.

This is an issue when working in an agent environment that the token
usage can drastically increase beyond the initial value set causing
unexpected behavior.
This commit is contained in:
Ayo Ayibiowu 2024-06-27 22:28:59 +02:00 committed by GitHub
parent 2a0d6788f7
commit c6f700b7cb
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
libs/community/langchain_community/chat_models/litellm.py
View File

@ -191,7 +191,7 @@ class ChatLiteLLM(BaseChatModel):
n: int = 1
"""Number of chat completions to generate for each prompt. Note that the API may
not return the full n completions if duplicates are generated."""
max_tokens: int = 256
max_tokens: Optional[int] = None
max_retries: int = 6