community[patch]: Invoke callback prior to yielding token fix [DeepInfra] (#20427)

- [x] **PR title**: community[patch]: Invoke callback prior to yielding
token fix for [DeepInfra]


- [x] **PR message**: 
- **Description:** Invoke callback prior to yielding token in stream
method in [DeepInfra]
    - **Issue:** https://github.com/langchain-ai/langchain/issues/16913
    - **Dependencies:** None
    - **Twitter handle:** @bolun_zhang

If no one reviews your PR within a few days, please @-mention one of
baskaryan, efriis, eyurtsev, hwchase17.
pull/20055/head
balloonio 2 months ago committed by GitHub
parent 450c458f8f
commit b66a4f48fa
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

@ -155,9 +155,9 @@ class DeepInfra(LLM):
for line in _parse_stream(response.iter_lines()):
chunk = _handle_sse_line(line)
if chunk:
yield chunk
if run_manager:
run_manager.on_llm_new_token(chunk.text)
yield chunk
async def _astream(
self,
@ -174,9 +174,9 @@ class DeepInfra(LLM):
async for line in _parse_stream_async(response.content):
chunk = _handle_sse_line(line)
if chunk:
yield chunk
if run_manager:
await run_manager.on_llm_new_token(chunk.text)
yield chunk
def _parse_stream(rbody: Iterator[bytes]) -> Iterator[str]:

Loading…
Cancel
Save