Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-11-06 03:20:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix typo (#3728)

Browse Source
This commit is contained in:
Alan Cha 2023-04-28 16:01:09 -04:00 committed by GitHub
parent 5042bd40d3
commit e3b7a20454
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/modules/indexes/text_splitters/examples/tiktoken.ipynb
View File

@ -6,7 +6,7 @@
"metadata": {}, "metadata": {},
"source": [ "source": [
"# tiktoken (OpenAI) Length Function\n", "# tiktoken (OpenAI) Length Function\n",
"You can also use tiktoken, a open source tokenizer package from OpenAI to estimate tokens used. Will probably be more accurate for their models.\n", "You can also use tiktoken, an open source tokenizer package from OpenAI to estimate tokens used. Will probably be more accurate for their models.\n",
"\n", "\n",
"1. How the text is split: by character passed in\n", "1. How the text is split: by character passed in\n",
"2. How the chunk size is measured: by `tiktoken` tokenizer" "2. How the chunk size is measured: by `tiktoken` tokenizer"