langchain/libs
Juan Daza 4236ae3851
Added Streaming Capability to SageMaker LLMs (#10535)
This PR adds the ability to declare a Streaming response in the
SageMaker LLM by leveraging the `invoke_endpoint_with_response_stream`
capability in `boto3`. It is heavily based on the AWS Blog Post
announcement linked
[here](https://aws.amazon.com/blogs/machine-learning/elevating-the-generative-ai-experience-introducing-streaming-support-in-amazon-sagemaker-hosting/).

It does not add any additional dependencies since it uses the existing
`boto3` version.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-10-05 11:08:43 -07:00
..
experimental Add SymbolicMathChain to experiment in preparation for deprecation (#11129) 2023-10-05 13:54:43 -04:00
langchain Added Streaming Capability to SageMaker LLMs (#10535) 2023-10-05 11:08:43 -07:00