You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
langchain/libs/community/langchain_community
Davide Menini f7042321f1
community[patch]: gather token usage info in BedrockChat during generation (#19127)
This PR allows to calculate token usage for prompts and completion
directly in the generation method of BedrockChat. The token usage
details are then returned together with the generations, so that other
downstream tasks can access them easily.

This allows to define a callback for tokens tracking and cost
calculation, similarly to what happens with OpenAI (see
[OpenAICallbackHandler](https://api.python.langchain.com/en/latest/_modules/langchain_community/callbacks/openai_info.html#OpenAICallbackHandler).
I plan on adding a BedrockCallbackHandler later.
Right now keeping track of tokens in the callback is already possible,
but it requires passing the llm, as done here:
https://how.wtf/how-to-count-amazon-bedrock-anthropic-tokens-with-langchain.html.
However, I find the approach of this PR cleaner.

Thanks for your reviews. FYI @baskaryan, @hwchase17

---------

Co-authored-by: taamedag <Davide.Menini@swisscom.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
6 months ago
..
adapters
agent_toolkits community[minor]: Update Azure Cognitive Services to Azure AI Services (#19488) 6 months ago
callbacks community[patch] : [Fiddler] ensure dataset is not added if model is present (#19293) 6 months ago
chat_loaders
chat_message_histories
chat_models community[patch]: gather token usage info in BedrockChat during generation (#19127) 6 months ago
docstore
document_compressors
document_loaders community[patch]: refactor the type hint of `file_path` in `UnstructuredAPIFileLoader` class (#18839) 6 months ago
document_transformers
embeddings Update text2vec.py (#19657) 6 months ago
example_selectors
graphs community[minor]: Add the option to omit schema refresh in Neo4jGraph (#19654) 6 months ago
indexes
llms community[patch]: gather token usage info in BedrockChat during generation (#19127) 6 months ago
output_parsers
retrievers
storage
tools community[minor]: Update Azure Cognitive Services to Azure AI Services (#19488) 6 months ago
utilities community[minor]: Add support for non-file-based Document Loaders in PebbloSafeLoader (#19574) 6 months ago
utils community[patch], mongodb[patch]: Stop spamming SIMD import warnings (#19531) 6 months ago
vectorstores community[minor]: Add VDMS vectorstore (#19551) 6 months ago
__init__.py
cache.py community[patch]: GPTCache pydantic validation error on lookup (#19427) 6 months ago
py.typed