Commit Graph

17 Commits (f1e13cfcc7e0f2a9015c562eede167b76b1d60fc)

Author SHA1 Message Date
Logan Kilpatrick f1e13cfcc7
Misc updates (#1022) 4 months ago
Gabor Cselle 2c441ab9a2
Migrate all notebooks to API V1 (#914)
Co-authored-by: ayush rajgor <ayushrajgorar@gmail.com>
4 months ago
simonpfish 6c00ce2ff5 Remove unnecessary cell outputs 7 months ago
Roger Zurawicki fc5af842a2
Add tiktoken-rs to third-party library (#791) 7 months ago
Simón Fishman c2959fd60b
[tiktoken_counting] fix tokenizer name (#741) 8 months ago
Eliah Kagan 63f95154b1
Add Tiktokenizer link in "How to count tokens" (#604)
This adds a link to Tiktokenizer webapp as another tool, in
addition to the OpenAI Tokenizer.
9 months ago
richzw 964dd85ed3
add tiktoken-go to How_to_count_tokens_with_tiktoken.ipynb (#605) 10 months ago
Ted Sanders de3bd58434
Ted/update token counting 0613 (#541)
* updates tiktoken counting function

* slight rewording

* updates ChatGPT guide with new models
11 months ago
Avery Khoo 26978e9ed1
Update How_to_count_tokens_with_tiktoken.ipynb (#511)
Add support for gpt-3.5-*-0613 models
11 months ago
aiqinxuancai 23bd266fef
Update How_to_count_tokens_with_tiktoken.ipynb (#368)
* Update How_to_count_tokens_with_tiktoken.ipynb

add c# tiktoken lib.

* Update How_to_count_tokens_with_tiktoken.ipynb

fix format.

* Update How_to_count_tokens_with_tiktoken.ipynb

fix format.
1 year ago
jiangying 4d59bafe25
add jtokkit to How_to_count_tokens_with_tiktoken.ipynb 1 year ago
dmitry-brazhenko be1f1187bd added sharptoken as exampel 1 year ago
Ted Sanders 7622aa1d20 updates description of token counting function 1 year ago
Ted Sanders b45d2b2346 updates token counting guide 1 year ago
Ted Sanders a101157740 updates token counting function to work with gpt-4 1 year ago
Ted Sanders 8cd6b0f53b adds examples of counting tokens for the ChatGPT API 1 year ago
Ted Sanders 5050739cf1 adds tiktoken example 1 year ago