mirror of https://github.com/hwchase17/langchain
google-vertexai[patch]: Fixed SafetySettings handling in streaming API in VertexAI (#17278)
The streaming API doesn't separate safety_settings from the generation_config payload. As the result the following error is observed when using `stream` API. The functionality is correct with `invoke` API. The fix separates the `safety_settings` from params and sets it as argument to the `send_message` method. ``` ERROR: Unknown field for GenerationConfig: safety_settings Traceback (most recent call last): File "/Users/user/Library/Caches/pypoetry/virtualenvs/chatbot-worker-main-Ju-qIM-X-py3.12/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 250, in stream raise e File "/Users/user/Library/Caches/pypoetry/virtualenvs/chatbot-worker-main-Ju-qIM-X-py3.12/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 234, in stream for chunk in self._stream( File "/Users/user/Library/Caches/pypoetry/virtualenvs/chatbot-worker-main-Ju-qIM-X-py3.12/lib/python3.12/site-packages/langchain_google_vertexai/chat_models.py", line 501, in _stream for response in responses: File "/Users/user/Library/Caches/pypoetry/virtualenvs/chatbot-worker-main-Ju-qIM-X-py3.12/lib/python3.12/site-packages/vertexai/generative_models/_generative_models.py", line 921, in _send_message_streaming for chunk in stream: File "/Users/user/Library/Caches/pypoetry/virtualenvs/chatbot-worker-main-Ju-qIM-X-py3.12/lib/python3.12/site-packages/vertexai/generative_models/_generative_models.py", line 514, in _generate_content_streaming request = self._prepare_request( ^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/Library/Caches/pypoetry/virtualenvs/chatbot-worker-main-Ju-qIM-X-py3.12/lib/python3.12/site-packages/vertexai/generative_models/_generative_models.py", line 256, in _prepare_request gapic_generation_config = gapic_content_types.GenerationConfig( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/Library/Caches/pypoetry/virtualenvs/chatbot-worker-main-Ju-qIM-X-py3.12/lib/python3.12/site-packages/proto/message.py", line 576, in __init__ raise ValueError( ValueError: Unknown field for GenerationConfig: safety_settings ``` --------- Co-authored-by: Erick Friis <erick@langchain.dev>pull/17077/head^2
parent
b18c6ab9ad
commit
96b5711a0c
Loading…
Reference in New Issue