"LangChain provides streaming support for LLMs. Currently, we support streaming for the `OpenAI`, `ChatOpenAI`, and `Anthropic` implementations, but streaming support for other LLM implementations is on the roadmap. To utilize streaming, use a [`CallbackHandler`](https://github.com/hwchase17/langchain/blob/master/langchain/callbacks/base.py) that implements `on_llm_new_token`. In this example, we are using [`StreamingStdOutCallbackHandler`]()."
"LangChain provides streaming support for LLMs. Currently, we support streaming for the `OpenAI`, `ChatOpenAI`, and `ChatAnthropic` implementations, but streaming support for other LLM implementations is on the roadmap. To utilize streaming, use a [`CallbackHandler`](https://github.com/hwchase17/langchain/blob/master/langchain/callbacks/base.py) that implements `on_llm_new_token`. In this example, we are using [`StreamingStdOutCallbackHandler`]()."
"LLMResult(generations=[[Generation(text='\\n\\nQ: What did the fish say when it hit the wall?\\nA: Dam!', generation_info={'finish_reason': None, 'logprobs': None})]], llm_output={'token_usage': {}, 'model_name': 'text-davinci-003'})"
"LLMResult(generations=[[Generation(text='\\n\\nQ: What did the fish say when it hit the wall?\\nA: Dam!', generation_info={'finish_reason': 'stop', 'logprobs': None})]], llm_output={'token_usage': {}, 'model_name': 'text-davinci-003'})"
]
},
"execution_count": 4,
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
@ -135,7 +135,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 4,
"id": "22665f16-e05b-473c-a4bd-ad75744ea024",
"metadata": {
"tags": []
@ -199,12 +199,12 @@
"id": "909ae48b-0f07-4990-bbff-e627f706c93e",
"metadata": {},
"source": [
"Here is an example with the `Anthropic` LLM implementation, which uses their `claude` model."
"Here is an example with the `ChatAnthropic` chat model implementation, which uses their `claude` model."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 5,
"id": "eadae4ba-9f21-4ec8-845d-dd43b0edc2dc",
"metadata": {
"tags": []
@ -214,38 +214,26 @@
"name": "stdout",
"output_type": "stream",
"text": [
" Here is my attempt at a song about sparkling water:\n",
"\n",
"Sparkling water, bubbles so bright,\n",
"\n",
"Fizzing and popping in the light.\n",
"\n",
"No sugar or calories, a healthy delight,\n",
"\n",
"Sparkling water, refreshing and light.\n",
"\n",
"Carbonation that tickles the tongue,\n",
"\n",
"In flavors of lemon and lime unsung.\n",
"\n",
"Sparkling water, a drink quite all right,\n",
"\n",
"Bubbles sparkling in the light."
"Sparkling water, bubbles so bright, \n",
"Dancing in the glass with delight.\n",
"Refreshing and crisp, a fizzy delight,\n",
"Quenching my thirst with each sip I take.\n",
"The carbonation tickles my tongue,\n",
"As the refreshing water song is sung.\n",
"Lime or lemon, a citrus twist,\n",
"Makes sparkling water such a bliss.\n",
"Healthy and hydrating, a drink so pure,\n",
"Sparkling water, always alluring.\n",
"Bubbles ascending in a stream, \n",
"Sparkling water, you're my dream!"
]
},
{
"data": {
"text/plain": [
"'\\nSparkling water, bubbles so bright,\\n\\nFizzing and popping in the light.\\n\\nNo sugar or calories, a healthy delight,\\n\\nSparkling water, refreshing and light.\\n\\nCarbonation that tickles the tongue,\\n\\nIn flavors of lemon and lime unsung.\\n\\nSparkling water, a drink quite all right,\\n\\nBubbles sparkling in the light.'"