langchain

Commit Graph

Author	SHA1	Message	Date
Michael Haddad	c6b27b3692	add konko chat_model files (#10267 ) _Thank you to the LangChain team for the great project and in advance for your review. Let me know if I can provide any other additional information or do things differently in the future to make your lives easier 🙏 _ @hwchase17 please let me know if you're not the right person to review 😄 This PR enables LangChain to access the Konko API via the chat_models API wrapper. Konko API is a fully managed API designed to help application developers: 1. Select the right LLM(s) for their application 2. Prototype with various open-source and proprietary LLMs 3. Move to production in-line with their security, privacy, throughput, latency SLAs without infrastructure set-up or administration using Konko AI's SOC 2 compliant infrastructure _Note on integration tests:_ We added 14 integration tests. They will all fail unless you export the right API keys. 13 will pass with a KONKO_API_KEY provided and the other one will pass with a OPENAI_API_KEY provided. When both are provided, all 14 integration tests pass. If you would like to test this yourself, please let me know and I can provide some temporary keys. ### Installation and Setup 1. First you'll need an API key 2. Install Konko AI's Python SDK 1. Enable a Python3.8+ environment `pip install konko` 3. Set API Keys Option 1: Set Environment Variables You can set environment variables for 1. KONKO_API_KEY (Required) 2. OPENAI_API_KEY (Optional) In your current shell session, use the export command: `export KONKO_API_KEY={your_KONKO_API_KEY_here}` `export OPENAI_API_KEY={your_OPENAI_API_KEY_here} #Optional` Alternatively, you can add the above lines directly to your shell startup script (such as .bashrc or .bash_profile for Bash shell and .zshrc for Zsh shell) to have them set automatically every time a new shell session starts. Option 2: Set API Keys Programmatically If you prefer to set your API keys directly within your Python script or Jupyter notebook, you can use the following commands: ```python konko.set_api_key('your_KONKO_API_KEY_here') konko.set_openai_api_key('your_OPENAI_API_KEY_here') # Optional ``` ### Calling a model Find a model on the [[Konko Introduction page](https://docs.konko.ai/docs#available-models)](https://docs.konko.ai/docs#available-models) For example, for this [[LLama 2 model](https://docs.konko.ai/docs/meta-llama-2-13b-chat)](https://docs.konko.ai/docs/meta-llama-2-13b-chat). The model id would be: `"meta-llama/Llama-2-13b-chat-hf"` Another way to find the list of models running on the Konko instance is through this [[endpoint](https://docs.konko.ai/reference/listmodels)](https://docs.konko.ai/reference/listmodels). From here, we can initialize our model: ```python chat_instance = ChatKonko(max_tokens=10, model = 'meta-llama/Llama-2-13b-chat-hf') ``` And run it: ```python msg = HumanMessage(content="Hi") chat_response = chat_instance([msg]) ```	12 months ago
Christoph Grotz	5a4ce9ef2b	VertexAI now allows to tune codey models (#10367 ) Description: VertexAI now supports to tune codey models, I adapted the Vertex AI LLM wrapper accordingly https://cloud.google.com/vertex-ai/docs/generative-ai/models/tune-code-models	12 months ago
William FH	1b0eebe1e3	Support multiple errors (#10376 ) in on_retry	12 months ago
bsenst	2423f7f3b4	add missing verb (#10371 )	12 months ago
Bagatur	d2d11ccf63	bump 285 (#10373 )	12 months ago
William FH	46e9abdc75	Add progress bar + runner fixes (#10348 ) - Add progress bar to eval runs - Use thread pool for concurrency - Update some error messages - Friendlier project name - Print out quantiles of the final stats Closes LS-902	12 months ago
Leonid Ganeline	0672533b3e	docs: fix `tools/sqlite` page (#10258 ) The `/docs/integrations/tools/sqlite` page is not about the tool integrations. I've moved it into `/docs/use_cases/sql/sqlite`. `vercel.json` modified As a result two pages now under the `/docs/use_cases/sql/` folder. So the `sql` root page moved down together with `sqlite` page.	12 months ago
Leonid Ganeline	f5d08be477	docs: `portkey` update (#10261 ) Added the `Portkey` description. Fixed a title in the nested document (and nested navbar).	12 months ago
Mateusz Wosinski	69fe0621d4	Merge branch 'master' into deepsense/text-to-speech	12 months ago
C Mazzoni	01e9d7902d	Update tool.py (#10203 ) Fixed the description of tool QuerySQLCheckerTool, the last line of the string description had the old name of the tool 'sql_db_query', this caused the models to sometimes call the non-existent tool The issue was not numerically identified. No dependencies	12 months ago
stopdropandrew	28de8d132c	Change StructuredTool's ainvoke to await (#10300 ) Fixes #10080. StructuredTool's `ainvoke` doesn't `await`.	12 months ago
Leonid Ganeline	fdba711d28	docs `integrations/embeddings` consistency (#10302 ) Updated `integrations/embeddings`: fixed titles; added links, descriptions Updated `integrations/providers`.	12 months ago
Leonid Ganeline	1b3ea1eeb4	docstrings: `chat_loaders` (#10307 ) Updated docstrings. Made them consistent across the module.	12 months ago
Bagatur	8826293c88	Add multilingual data anon chain (#10346 )	12 months ago
Greg Richardson	300559695b	Supabase vector self querying retriever (#10304 ) ## Description Adds Supabase Vector as a self-querying retriever. - Designed to be backwards compatible with existing `filter` logic on `SupabaseVectorStore`. - Adds new filter `postgrest_filter` to `SupabaseVectorStore` `similarity_search()` methods - Supports entire PostgREST [filter query language](https://postgrest.org/en/stable/references/api/tables_views.html#read) (used by self-querying retriever, but also works as an escape hatch for more query control) - `SupabaseVectorTranslator` converts Langchain filter into the above PostgREST query - Adds Jupyter Notebook for the self-querying retriever - Adds tests ## Tag maintainer @hwchase17 ## Twitter handle [@ggrdson](https://twitter.com/ggrdson)	12 months ago
Tze Min	20c742d8a2	Enhancement: add parameter boto3_session for AWS DynamoDB cross account use cases (#10326 ) - Description: to allow boto3 assume role for AWS cross account use cases to read and update the chat history, - Issue: use case I faced in my company, - Dependencies: no - Tag maintainer: @baskaryan , - Twitter handle: @tmin97 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago
kcocco	b1d40b8626	Fix colab link(missing graph in url) and comment to match the code fo… (#10344 ) - Description: Fixing Colab broken link and comment correction to align with the code that uses Warren Buffet for wiki query - Issue: None open - Dependencies: none - Tag maintainer: n/a - Twitter handle: Not a PR change but: kcocco	12 months ago
Bagatur	49e0c83126	Split LCEL cookbook (#10342 )	12 months ago
Bagatur	41a2548611	Fix presidio docs Colab links	12 months ago
Bagatur	1d2b6c3c67	Reorganize presidio anonymization docs	12 months ago
maks-operlejn-ds	274c3dc3a8	Multilingual anonymization (#10327 ) ### Description Add multiple language support to Anonymizer PII detection in Microsoft Presidio relies on several components - in addition to the usual pattern matching (e.g. using regex), the analyser uses a model for Named Entity Recognition (NER) to extract entities such as: - `PERSON` - `LOCATION` - `DATE_TIME` - `NRP` - `ORGANIZATION` [[Source]](https://github.com/microsoft/presidio/blob/main/presidio-analyzer/presidio_analyzer/predefined_recognizers/spacy_recognizer.py) To handle NER in specific languages, we utilize unique models from the `spaCy` library, recognized for its extensive selection covering multiple languages and sizes. However, it's not restrictive, allowing for integration of alternative frameworks such as [Stanza](https://microsoft.github.io/presidio/analyzer/nlp_engines/spacy_stanza/) or [transformers](https://microsoft.github.io/presidio/analyzer/nlp_engines/transformers/) when necessary. ### Future works - automatic language detection - instead of passing the language as a parameter in `anonymizer.anonymize`, we could detect the language/s beforehand and then use the corresponding NER model. We have discussed this internally and @mateusz-wosinski-ds will look into a standalone language detection tool/chain for LangChain 😄 ### Twitter handle @deepsense_ai / @MaksOpp ### Tag maintainer @baskaryan @hwchase17 @hinthornw	12 months ago
mateusz.wosinski	f23fed34e8	Added TYPE_CHECKING	12 months ago
mateusz.wosinski	ff1c6de86c	TYPE_CHECKING added	12 months ago
mateusz.wosinski	868db99b17	Merge branch 'master' into deepsense/text-to-speech	12 months ago
Ofer Mendelevitch	a9eb7c6cfc	Adding Self-querying for Vectara (#10332 ) - Description: Adding support for self-querying to Vectara integration - Issue: per customer request - Tag maintainer: @rlancemartin @baskaryan - Twitter handle: @ofermend Also updated some documentation, added self-query testing, and a demo notebook with self-query example.	12 months ago
Bagatur	25ec655e4f	supabase embedding usage fix (#10335 ) Should be calling Embeddings.embed_query instead of embed_documents when searching	12 months ago
Bagatur	f0ccce76fe	nuclia db nit (#10334 )	12 months ago
Bagatur	205f406485	nuclia nb nit (#10331 )	12 months ago
Bagatur	672907bbbb	bump 284 (#10330 )	12 months ago
maks-operlejn-ds	f747e76b73	Fixed link to colab notebook (#10320 ) small fix to anonymizer documentation	12 months ago
maks-operlejn-ds	4cc4534d81	Data deanonymization (#10093 ) ### Description The feature for pseudonymizing data with ability to retrieve original text (deanonymization) has been implemented. In order to protect private data, such as when querying external APIs (OpenAI), it is worth pseudonymizing sensitive data to maintain full privacy. But then, after the model response, it would be good to have the data in the original form. I implemented the `PresidioReversibleAnonymizer`, which consists of two parts: 1. anonymization - it works the same way as `PresidioAnonymizer`, plus the object itself stores a mapping of made-up values to original ones, for example: ``` { "PERSON": { "<anonymized>": "<original>", "John Doe": "Slim Shady" }, "PHONE_NUMBER": { "111-111-1111": "555-555-5555" } ... } ``` 2. deanonymization - using the mapping described above, it matches fake data with original data and then substitutes it. Between anonymization and deanonymization user can perform different operations, for example, passing the output to LLM. ### Future works - instance anonymization - at this point, each occurrence of PII is treated as a separate entity and separately anonymized. Therefore, two occurrences of the name John Doe in the text will be changed to two different names. It is therefore worth introducing support for full instance detection, so that repeated occurrences are treated as a single object. - better matching and substitution of fake values for real ones - currently the strategy is based on matching full strings and then substituting them. Due to the indeterminism of language models, it may happen that the value in the answer is slightly changed (e.g. John Doe -> John or Main St, New York -> New York) and such a substitution is then no longer possible. Therefore, it is worth adjusting the matching for your needs. - Q&A with anonymization - when I'm done writing all the functionality, I thought it would be a cool resource in documentation to write a notebook about retrieval from documents using anonymization. An iterative process, adding new recognizers to fit the data, lessons learned and what to look out for ### Twitter handle @deepsense_ai / @MaksOpp --------- Co-authored-by: MaksOpp <maks.operlejn@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago
Bagatur	67696fe3ba	Add myscale vector sql retriever chain (#10305 )	12 months ago
Bagatur	f4f9254dad	Move Myscale SQL vector retrieval nb	12 months ago
刘方瑞	890ed775a3	Resolve: VectorSearch enabled SQLChain? (#10177 ) Squashed from #7454 with updated features We have separated the `SQLDatabseChain` from `VectorSQLDatabseChain` and put everything into `experimental/`. Below is the original PR message from #7454. ------- We have been working on features to fill up the gap among SQL, vector search and LLM applications. Some inspiring works like self-query retrievers for VectorStores (for example [Weaviate](https://python.langchain.com/en/latest/modules/indexes/retrievers/examples/weaviate_self_query.html) and [others](https://python.langchain.com/en/latest/modules/indexes/retrievers/examples/self_query.html)) really turn those vector search databases into a powerful knowledge base! 🚀🚀 We are thinking if we can merge all in one, like SQL and vector search and LLMChains, making this SQL vector database memory as the only source of your data. Here are some benefits we can think of for now, maybe you have more 👀: With ALL data you have: since you store all your pasta in the database, you don't need to worry about the foreign keys or links between names from other data source. Flexible data structure: Even if you have changed your schema, for example added a table, the LLM will know how to JOIN those tables and use those as filters. SQL compatibility: We found that vector databases that supports SQL in the marketplace have similar interfaces, which means you can change your backend with no pain, just change the name of the distance function in your DB solution and you are ready to go! ### Issue resolved: - [Feature Proposal: VectorSearch enabled SQLChain?](https://github.com/hwchase17/langchain/issues/5122) ### Change made in this PR: - An improved schema handling that ignore `types.NullType` columns - A SQL output Parser interface in `SQLDatabaseChain` to enable Vector SQL capability and further more - A Retriever based on `SQLDatabaseChain` to retrieve data from the database for RetrievalQAChains and many others - Allow `SQLDatabaseChain` to retrieve data in python native format - Includes PR #6737 - Vector SQL Output Parser for `SQLDatabaseChain` and `SQLDatabaseChainRetriever` - Prompts that can implement text to VectorSQL - Corresponding unit-tests and notebook ### Twitter handle: - @MyScaleDB ### Tag Maintainer: Prompts / General: @hwchase17, @baskaryan DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev ### Dependencies: No dependency added	12 months ago
Bagatur	849e345371	Bagatur/nuclia vector (#10301 )	12 months ago
Bagatur	0c760f184c	Update NucliaDB vecstore deps	12 months ago
Eric BREHAULT	19b4ecdc39	Implement NucliaDB vector store (#10236 ) # Description This pull request allows to use the [NucliaDB](https://docs.nuclia.dev/docs/docs/nucliadb/intro) as a vector store in LangChain. It works with both a [local NucliaDB instance](https://docs.nuclia.dev/docs/docs/nucliadb/deploy/basics) or with [Nuclia Cloud](https://nuclia.cloud). # Dependencies It requires an up-to-date version of the `nuclia` Python package. @rlancemartin, @eyurtsev, @hinthornw, please review it when you have a moment :) Note: our Twitter handler is `@NucliaAI`	12 months ago
cccs-eric	b64a443f72	Fix SQL search_path for Trino query engine (#10248 ) This PR replaces the generic `SET search_path TO` statement by `USE` for the Trino dialect since Trino does not support `SET search_path`. Official Trino documentation can be found [here](https://trino.io/docs/current/sql/use.html). With this fix, the `SQLdatabase` will now be able to set the current schema and execute queries using the Trino engine. It will use the catalog set as default by the connection uri.	12 months ago
Bagatur	1fb7bdd595	Split sql use case docs (#10257 ) Split sql use case into directory so we can add other structured data pages	12 months ago
Bagatur	763212eafd	Add use case nb position (#10299 )	12 months ago
Ikko Eltociear Ashimine	ea5d29a702	Update amazon_comprehend_chain.ipynb (#10246 ) Huggingface, HuggingFace -> Hugging Face	12 months ago
Brian Antonelli	4df101cf77	Don't hardcode PGVector distance strategies (#10265 ) - Description: Remove hardcoded/duplicated distance strategies in the PGVector store. - Issue: NA - Dependencies: NA - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: @archmonkeymojo --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago
captivus	86cb9da735	Updated Additional Resources section of documentation (#10260 ) - Description: Updated Additional Resources section of documentation and added to YouTube videos with excellent playlist of Langchain content from Sam Witteveen - Issue: None -- updating documentation - Dependencies: None - Tag maintainer: @baskaryan	12 months ago
JaéGeR	b8669b249e	Added Hugging face inference api (#10280 ) Embed documents without locally downloading the HF model --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago
Ilya	6e6f15df24	Add strip text splits flag (#10295 ) #10085 --------- Co-authored-by: codesee-maps[bot] <86324825+codesee-maps[bot]@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago
Randy	1690013711	Doc: openai_functions_agent.mdx import (#10282 ) Fix the import in docmention --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago
William FH	13c5951e26	Add LCEL cookbook examples (#10290 ) 1. For passing config to runnable lambda 2. For branching and merging	12 months ago
ParamdeepSinghShorthillsAI	3cc242b591	Update rwkv.py import error (#10293 ) I have updated the code to ensure consistent error handling for ImportError. Instead of relying on ValueError as before, I've followed the standard practice of raising ImportError while also including detailed error messages. This modification improves code clarity and explicitly indicates that any issues are related to module imports.	12 months ago
Pihplipe Oegr	bce38b7163	Add notebook example to use sqlite-vss as a vector store. (#10292 ) Follow-up PR for https://github.com/langchain-ai/langchain/pull/10047, simply adding a notebook quickstart example for the vector store with SQLite, using the class SQLiteVSS. Maintainer tag @baskaryan Co-authored-by: Philippe Oger <philippe.oger@adevinta.com>	12 months ago
Tomaz Bratanic	db73c9d5b5	Diffbot Graph Transformer / Neo4j Graph document ingestion (#9979 ) Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago

1 2 3 4 5 ...

4485 Commits (1835624badca65d598530530a17691fa49bd9bff) All Branches Search

4485 Commits (1835624badca65d598530530a17691fa49bd9bff)

All Branches