Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-11-06 03:20:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

langchain_nvidia_ai_endpoints[patch]: Invoke callback prior to yielding token (#18271)

Browse Source 
## PR title
langchain_nvidia_ai_endpoints[patch]: Invoke callback prior to yielding

## PR message
**Description:** Invoke callback prior to yielding token in _stream and
_astream methods for nvidia_ai_endpoints.
**Issue:** https://github.com/langchain-ai/langchain/issues/16913
**Dependencies:** None
This commit is contained in:
William De Vena 2024-02-28 19:10:57 +01:00 committed by GitHub
parent b4f6066a57
commit 7ac74f291e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
libs/partners/nvidia-ai-endpoints/langchain_nvidia_ai_endpoints/chat_models.py
View File

@ -161,9 +161,9 @@ class ChatNVIDIA(nvidia_ai_endpoints._NVIDIAClient, SimpleChatModel):
inputs = self.custom_preprocess(messages)
for response in self.get_stream(inputs=inputs, stop=stop, **kwargs):
chunk = self._get_filled_chunk(self.custom_postprocess(response))
yield chunk
if run_manager:
run_manager.on_llm_new_token(chunk.text, chunk=chunk)
yield chunk
async def _astream(
self,
@ -175,9 +175,9 @@ class ChatNVIDIA(nvidia_ai_endpoints._NVIDIAClient, SimpleChatModel):
inputs = self.custom_preprocess(messages)
async for response in self.get_astream(inputs=inputs, stop=stop, **kwargs):
chunk = self._get_filled_chunk(self.custom_postprocess(response))
yield chunk
if run_manager:
await run_manager.on_llm_new_token(chunk.text, chunk=chunk)
yield chunk
def custom_preprocess(
self, msg_list: Sequence[BaseMessage]