From fea69be4cdc2dc6c8eb64de5ed84750b8ef7457a Mon Sep 17 00:00:00 2001 From: Eugene Yan Date: Sat, 15 Apr 2023 13:01:02 -0700 Subject: [PATCH] Update the section token count from 1,000 to 1,600 This resolves the mismatch between the values in markdown cell and code cell. --- examples/Embedding_Wikipedia_articles_for_search.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/Embedding_Wikipedia_articles_for_search.ipynb b/examples/Embedding_Wikipedia_articles_for_search.ipynb index 9da41900..69b609da 100644 --- a/examples/Embedding_Wikipedia_articles_for_search.ipynb +++ b/examples/Embedding_Wikipedia_articles_for_search.ipynb @@ -411,7 +411,7 @@ "- Shorter sections allow more sections to be retrieved, which may help with recall\n", "- Overlapping sections may help prevent answers from being cut by section boundaries\n", "\n", - "Here, we'll use a simple approach and limit sections to 1,000 tokens each, recursively halving any sections that are too long. To avoid cutting in the middle of useful sentences, we'll split along paragraph boundaries when possible." + "Here, we'll use a simple approach and limit sections to 1,600 tokens each, recursively halving any sections that are too long. To avoid cutting in the middle of useful sentences, we'll split along paragraph boundaries when possible." ] }, {