fix: prevent adding an empty string to the result queue in AsyncIteratorCallbackHandler (#7180)

- Description: Modify the code for
AsyncIteratorCallbackHandler.on_llm_new_token to ensure that it does not
add an empty string to the result queue.
- Tag maintainer: @agola11

When using AsyncIteratorCallbackHandler with OpenAIFunctionsAgent, if
the LLM response function_call instead of direct answer, the
AsyncIteratorCallbackHandler.on_llm_new_token would be called with empty
string.
see also: langchain.chat_models.openai.ChatOpenAI._generate

An alternative solution is to modify the
langchain.chat_models.openai.ChatOpenAI._generate and do not call the
run_manager.on_llm_new_token when the token is empty string.
I am not sure which solution is better.

@hwchase17

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
pull/7213/head
Shuqian 1 year ago committed by GitHub
parent db98c44f8f
commit 8045870a0f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -31,7 +31,8 @@ class AsyncIteratorCallbackHandler(AsyncCallbackHandler):
self.done.clear()
async def on_llm_new_token(self, token: str, **kwargs: Any) -> None:
self.queue.put_nowait(token)
if token is not None and token != "":
self.queue.put_nowait(token)
async def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
self.done.set()

Loading…
Cancel
Save