blob42/langchain
1
0
Fork 0
forked from Archives/langchain
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Allow no token limit for ChatGPT API (#1481)

Browse Source 
The endpoint default is inf if we don't specify max_tokens, so unlike
regular completion API, we don't need to calculate this based on the
prompt.
This commit is contained in:
kahkeng 2023-03-06 13:18:55 -08:00 committed by GitHub
parent 312c319d8b
commit df6865cd52
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
langchain/llms/openai.py
View File

@ -619,6 +619,9 @@ class OpenAIChat(BaseLLM, BaseModel):
if "stop" in params: if "stop" in params:
raise ValueError("`stop` found in both the input and default params.") raise ValueError("`stop` found in both the input and default params.")
params["stop"] = stop params["stop"] = stop
if params.get("max_tokens") == -1:
# for ChatGPT api, omitting max_tokens is equivalent to having no limit
del params["max_tokens"]
return messages, params return messages, params
def _generate( def _generate(