`fallback` definition (#10504)

I've added a definition to `fallback` and fixed couple misspells. It was
not really clear what is the "fallback".
pull/11361/head
Leonid Ganeline 9 months ago committed by GitHub
parent 5d6b83d9cf
commit 39316314fa
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -7,9 +7,11 @@
"source": [
"# Fallbacks\n",
"\n",
"When working with language models, you may often encounter issues from the underlying APIs, whether these be rate limiting or downtime. Therefore, as you go to move your LLM applications into production it becomes more and more important to safe guard against these. That's why we've introduced the concept of fallbacks.\n",
"When working with language models, you may often encounter issues from the underlying APIs, whether these be rate limiting or downtime. Therefore, as you go to move your LLM applications into production it becomes more and more important to safeguard against these. That's why we've introduced the concept of fallbacks. \n",
"\n",
"Crucially, fallbacks can be applied not only on the LLM level but on the whole runnable level. This is important because often times different models require different prompts. So if your call to OpenAI fails, you don't just want to send the same prompt to Anthropic - you probably want want to use a different prompt template and send a different version there."
"A **fallback** is an alternative plan that may be used in an emergency.\n",
"\n",
"Crucially, fallbacks can be applied not only on the LLM level but on the whole runnable level. This is important because often times different models require different prompts. So if your call to OpenAI fails, you don't just want to send the same prompt to Anthropic - you probably want to use a different prompt template and send a different version there."
]
},
{
@ -17,7 +19,7 @@
"id": "a6bb9ba9",
"metadata": {},
"source": [
"## Handling LLM API Errors\n",
"## Fallback for LLM API Errors\n",
"\n",
"This is maybe the most common use case for fallbacks. A request to an LLM API can fail for a variety of reasons - the API could be down, you could have hit rate limits, any number of things. Therefore, using fallbacks can help protect against these types of things.\n",
"\n",
@ -156,7 +158,7 @@
"id": "8d62241b",
"metadata": {},
"source": [
"## Fallbacks for Sequences\n",
"## Fallback for Sequences\n",
"\n",
"We can also create fallbacks for sequences, that are sequences themselves. Here we do that with two different models: ChatOpenAI and then normal OpenAI (which does not use a chat model). Because OpenAI is NOT a chat model, you likely want a different prompt."
]
@ -230,9 +232,9 @@
"id": "ec4685b4",
"metadata": {},
"source": [
"## Handling Long Inputs\n",
"## Fallback for Long Inputs\n",
"\n",
"One of the big limiting factors of LLMs in their context window. Usually you can count and track the length of prompts before sending them to an LLM, but in situations where that is hard/complicated you can fallback to a model with longer context length."
"One of the big limiting factors of LLMs is their context window. Usually, you can count and track the length of prompts before sending them to an LLM, but in situations where that is hard/complicated, you can fallback to a model with a longer context length."
]
},
{
@ -422,7 +424,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
"version": "3.10.12"
}
},
"nbformat": 4,

Loading…
Cancel
Save