Commit Graph

4524 Commits (534f1b63c5a6341c358ef4135ed15e865fe518e3)
 

Author SHA1 Message Date
olgavrou 534f1b63c5 Merge remote-tracking branch 'origin' into move_things_around 1 year ago
olgavrou 3d700aa654 merge from upstream/master 1 year ago
olgavrou 2dba4046fa update experimental poetry lock 1 year ago
olgavrou b78d672a43 merge from upstream/master 1 year ago
olgavrou 11f20cded1 move everything into experimental 1 year ago
Bagatur 8b5662473f
bump 286 (#10412) 1 year ago
Sam Partee 65e1606daa
Fix the RedisVectorStoreRetriever import (#10414)
As the title suggests.

Replace this entire comment with:
  - Description: Add a syntactic sugar import fix for #10186 
  - Issue: #10186 
  - Tag maintainer: @baskaryan 
  - Twitter handle: @Spartee
1 year ago
Sam Partee d09ef9eb52
Redis: Fix keys (#10413)
- Description: Fixes user issue with custom keys for ``from_texts`` and
``from_documents`` methods.
  - Issue: #10411 
  - Tag maintainer: @baskaryan 
  - Twitter handle: @spartee
1 year ago
John Mai ee3f950a67
Supported custom ernie_api_base & Implemented asynchronous for ErnieEmbeddings (#10398)
Description: Supported custom ernie_api_base & Implemented asynchronous
for ErnieEmbeddings
 - ernie_api_base:Support Ernie Service custom endpoints
 - Support asynchronous 

Issue: None
Dependencies: None
Tag maintainer:
Twitter handle: @JohnMai95
1 year ago
John Mai e0d45e6a09
Implemented MMR search for PGVector (#10396)
Description: Implemented MMR search for PGVector.
Issue: #7466
Dependencies: None
Tag maintainer: 
Twitter handle: @JohnMai95
1 year ago
Leonid Ganeline 90504fc499
`chat_loaders` refactoring (#10381)
Replaced unnecessary namespace renaming
`from langchain.chat_loaders import base as chat_loaders`
with
`from langchain.chat_loaders.base import BaseChatLoader, ChatSession` 
and simplified correspondent types.

@eyurtsev
1 year ago
Harrison Chase 40d9191955
runnable powered agent (#10407) 1 year ago
ColabDog 6ad6bb46c4
Feature/add deepeval (#10349)
Description: Adding `DeepEval` - which provides an opinionated framework
for testing and evaluating LLMs
Issue: Missing Deepeval
Dependencies: Optional DeepEval dependency
Tag maintainer: @baskaryan   (not 100% sure)
Twitter handle: https://twitter.com/ColabDog
1 year ago
eryk-dsai 675d57df50
New LLM integration: Ctranslate2 (#10400)
## Description:

I've integrated CTranslate2 with LangChain. CTranlate2 is a recently
popular library for efficient inference with Transformer models that
compares favorably to alternatives such as HF Text Generation Inference
and vLLM in
[benchmarks](https://hamel.dev/notes/llm/inference/03_inference.html).
1 year ago
Tarek Abouzeid ddd07001f3
adding language as parameter to NLTK text splitter (#10229)
- Description: 
Adding language as parameter to NLTK, by default it is only using
English. This will help using NLTK splitter for other languages. Change
is simple, via adding language as parameter to NLTKTextSplitter and then
passing it to nltk "sent_tokenize".
  
  - Issue: N/A
  
  - Dependencies: N/A

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
1 year ago
Markus Tretzmüller b3a8fc7cb1
enable serde retrieval qa with sources (#10132)
#3983 mentions serialization/deserialization issues with both
`RetrievalQA` & `RetrievalQAWithSourcesChain`.
`RetrievalQA` has already been fixed in #5818. 

Mimicing #5818, I added the logic for `RetrievalQAWithSourcesChain`.

---------

Co-authored-by: Markus Tretzmüller <markus.tretzmueller@cortecs.at>
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
zhanghexian 62fa2bc518
Add Vearch vectorstore (#9846)
---------

Co-authored-by: zhanghexian1 <zhanghexian1@jd.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
1 year ago
Jeremy Lai e93240f023
add where_document filter for chroma (#10214)
- Description: add where_document filter parameter in Chroma
- Issue: [10082](https://github.com/langchain-ai/langchain/issues/10082)
  - Dependencies: no
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
  - Twitter handle: no

@hwchase17

---------

Co-authored-by: Jeremy Lai <jeremy_lai@wiwynn.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur 7203c97e8f
Add redis self-query support (#10199) 1 year ago
Syed Ather Rizvi 4258c23867
Feature/adding csharp support to textsplitter (#10350)
**Description:** Adding C# language support for
`RecursiveCharacterTextSplitter`
**Issue:**   N/A
**Dependencies:** N/A

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Hugues 3e5a143625
Enhancements and bug fixes for `LLMonitorCallbackHandler` (#10297)
Hi @baskaryan,

I've made updates to LLMonitorCallbackHandler to address a few bugs
reported by users
These changes don't alter the fundamental behavior of the callback
handler.

Thanks you!

---------

Co-authored-by: vincelwt <vince@lyser.io>
1 year ago
captivus c902a1545b
Resolves issue DOC: Incorrect and confusing documentation of AIMessag… (#10379)
Resolves issue DOC: Incorrect and confusing documentation of
AIMessagePromptTemplate and HumanMessagePromptTemplate #10378

- Description: Revised docstrings to correctly and clearly document each
PromptTemplate
- Issue: #10378
- Dependencies: N/A
- Tag maintainer: @baskaryan

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Hamza Tahboub 8c0f391815
Implemented MMR search for Redis (#10140)
Description: Implemented MMR search for Redis. Pretty straightforward,
just using the already implemented MMR method on similarity
search–fetched docs.
Issue: #10059
Dependencies: None
Twitter handle: @hamza_tahboub

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur 5d8a689d5e
Add konko chat model (#10380) 1 year ago
Bagatur 0a86a70fe7 Merge branch 'master' into bagatur/add_konko_chat_model 1 year ago
Bagatur 9095dc69ac Konko fix dependency 1 year ago
Michael Haddad c6b27b3692
add konko chat_model files (#10267)
_Thank you to the LangChain team for the great project and in advance
for your review. Let me know if I can provide any other additional
information or do things differently in the future to make your lives
easier 🙏 _

@hwchase17 please let me know if you're not the right person to review 😄

This PR enables LangChain to access the Konko API via the chat_models
API wrapper.

Konko API is a fully managed API designed to help application
developers:

1. Select the right LLM(s) for their application
2. Prototype with various open-source and proprietary LLMs
3. Move to production in-line with their security, privacy, throughput,
latency SLAs without infrastructure set-up or administration using Konko
AI's SOC 2 compliant infrastructure

_Note on integration tests:_ 
We added 14 integration tests. They will all fail unless you export the
right API keys. 13 will pass with a KONKO_API_KEY provided and the other
one will pass with a OPENAI_API_KEY provided. When both are provided,
all 14 integration tests pass. If you would like to test this yourself,
please let me know and I can provide some temporary keys.

### Installation and Setup

1. **First you'll need an API key**
2. **Install Konko AI's Python SDK**
    1. Enable a Python3.8+ environment
    
    `pip install konko`
    
3.  **Set API Keys**
    
          **Option 1:** Set Environment Variables
    
    You can set environment variables for
    
    1. KONKO_API_KEY (Required)
    2. OPENAI_API_KEY (Optional)
    
    In your current shell session, use the export command:
    
    `export KONKO_API_KEY={your_KONKO_API_KEY_here}`
    `export OPENAI_API_KEY={your_OPENAI_API_KEY_here} #Optional`
    
Alternatively, you can add the above lines directly to your shell
startup script (such as .bashrc or .bash_profile for Bash shell and
.zshrc for Zsh shell) to have them set automatically every time a new
shell session starts.
    
    **Option 2:** Set API Keys Programmatically
    
If you prefer to set your API keys directly within your Python script or
Jupyter notebook, you can use the following commands:
    
    ```python
    konko.set_api_key('your_KONKO_API_KEY_here')
    konko.set_openai_api_key('your_OPENAI_API_KEY_here') # Optional
    
    ```
    

### Calling a model

Find a model on the [[Konko Introduction
page](https://docs.konko.ai/docs#available-models)](https://docs.konko.ai/docs#available-models)

For example, for this [[LLama 2
model](https://docs.konko.ai/docs/meta-llama-2-13b-chat)](https://docs.konko.ai/docs/meta-llama-2-13b-chat).
The model id would be: `"meta-llama/Llama-2-13b-chat-hf"`

Another way to find the list of models running on the Konko instance is
through this
[[endpoint](https://docs.konko.ai/reference/listmodels)](https://docs.konko.ai/reference/listmodels).

From here, we can initialize our model:

```python
chat_instance = ChatKonko(max_tokens=10, model = 'meta-llama/Llama-2-13b-chat-hf')

```

And run it:

```python
msg = HumanMessage(content="Hi")
chat_response = chat_instance([msg])

```
1 year ago
Christoph Grotz 5a4ce9ef2b
VertexAI now allows to tune codey models (#10367)
Description: VertexAI now supports to tune codey models, I adapted the
Vertex AI LLM wrapper accordingly
https://cloud.google.com/vertex-ai/docs/generative-ai/models/tune-code-models
1 year ago
William FH 1b0eebe1e3
Support multiple errors (#10376)
in on_retry
1 year ago
bsenst 2423f7f3b4
add missing verb (#10371) 1 year ago
Bagatur d2d11ccf63
bump 285 (#10373) 1 year ago
William FH 46e9abdc75
Add progress bar + runner fixes (#10348)
- Add progress bar to eval runs
- Use thread pool for concurrency
- Update some error messages
- Friendlier project name
- Print out quantiles of the final stats 

Closes LS-902
1 year ago
Leonid Ganeline 0672533b3e
docs: fix `tools/sqlite` page (#10258)
The `/docs/integrations/tools/sqlite` page is not about the tool
integrations.
I've moved it into `/docs/use_cases/sql/sqlite`. 
`vercel.json` modified
As a result two pages now under the `/docs/use_cases/sql/` folder. So
the `sql` root page moved down together with `sqlite` page.
1 year ago
Leonid Ganeline f5d08be477
docs: `portkey` update (#10261)
Added the `Portkey` description. Fixed a title in the nested document
(and nested navbar).
1 year ago
C Mazzoni 01e9d7902d
Update tool.py (#10203)
Fixed the description of tool QuerySQLCheckerTool, the last line of the
string description had the old name of the tool 'sql_db_query', this
caused the models to sometimes call the non-existent tool
The issue was not numerically identified.
No dependencies
1 year ago
stopdropandrew 28de8d132c
Change StructuredTool's ainvoke to await (#10300)
Fixes #10080. StructuredTool's `ainvoke` doesn't `await`.
1 year ago
Leonid Ganeline fdba711d28
docs `integrations/embeddings` consistency (#10302)
Updated `integrations/embeddings`: fixed titles; added links,
descriptions
Updated `integrations/providers`.
1 year ago
Leonid Ganeline 1b3ea1eeb4
docstrings: `chat_loaders` (#10307)
Updated docstrings. Made them consistent across the module.
1 year ago
Bagatur 8826293c88
Add multilingual data anon chain (#10346) 1 year ago
Greg Richardson 300559695b
Supabase vector self querying retriever (#10304)
## Description
Adds Supabase Vector as a self-querying retriever.

- Designed to be backwards compatible with existing `filter` logic on
`SupabaseVectorStore`.
- Adds new filter `postgrest_filter` to `SupabaseVectorStore`
`similarity_search()` methods
- Supports entire PostgREST [filter query
language](https://postgrest.org/en/stable/references/api/tables_views.html#read)
(used by self-querying retriever, but also works as an escape hatch for
more query control)
- `SupabaseVectorTranslator` converts Langchain filter into the above
PostgREST query
- Adds Jupyter Notebook for the self-querying retriever
- Adds tests

## Tag maintainer
@hwchase17

## Twitter handle
[@ggrdson](https://twitter.com/ggrdson)
1 year ago
Tze Min 20c742d8a2
Enhancement: add parameter boto3_session for AWS DynamoDB cross account use cases (#10326)
- Description: to allow boto3 assume role for AWS cross account use
cases to read and update the chat history,
  - Issue: use case I faced in my company,
  - Dependencies: no
  - Tag maintainer: @baskaryan ,
  - Twitter handle: @tmin97

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
kcocco b1d40b8626
Fix colab link(missing graph in url) and comment to match the code fo… (#10344)
- Description: Fixing Colab broken link and comment correction to align
with the code that uses Warren Buffet for wiki query
  - Issue: None open
  - Dependencies: none
  - Tag maintainer: n/a
  - Twitter handle: Not a PR change but: kcocco
1 year ago
Bagatur 49e0c83126
Split LCEL cookbook (#10342) 1 year ago
Bagatur 41a2548611 Fix presidio docs Colab links 1 year ago
Bagatur 1d2b6c3c67 Reorganize presidio anonymization docs 1 year ago
maks-operlejn-ds 274c3dc3a8
Multilingual anonymization (#10327)
### Description

Add multiple language support to Anonymizer

PII detection in Microsoft Presidio relies on several components - in
addition to the usual pattern matching (e.g. using regex), the analyser
uses a model for Named Entity Recognition (NER) to extract entities such
as:
- `PERSON`
- `LOCATION`
- `DATE_TIME`
- `NRP`
- `ORGANIZATION`


[[Source]](https://github.com/microsoft/presidio/blob/main/presidio-analyzer/presidio_analyzer/predefined_recognizers/spacy_recognizer.py)

To handle NER in specific languages, we utilize unique models from the
`spaCy` library, recognized for its extensive selection covering
multiple languages and sizes. However, it's not restrictive, allowing
for integration of alternative frameworks such as
[Stanza](https://microsoft.github.io/presidio/analyzer/nlp_engines/spacy_stanza/)
or
[transformers](https://microsoft.github.io/presidio/analyzer/nlp_engines/transformers/)
when necessary.

### Future works

- **automatic language detection** - instead of passing the language as
a parameter in `anonymizer.anonymize`, we could detect the language/s
beforehand and then use the corresponding NER model. We have discussed
this internally and @mateusz-wosinski-ds will look into a standalone
language detection tool/chain for LangChain 😄

### Twitter handle
@deepsense_ai / @MaksOpp

### Tag maintainer
@baskaryan @hwchase17 @hinthornw
1 year ago
Ofer Mendelevitch a9eb7c6cfc
Adding Self-querying for Vectara (#10332)
- Description: Adding support for self-querying to Vectara integration
  - Issue: per customer request
  - Tag maintainer: @rlancemartin @baskaryan 
  - Twitter handle: @ofermend 

Also updated some documentation, added self-query testing, and a demo
notebook with self-query example.
1 year ago
Bagatur 25ec655e4f
supabase embedding usage fix (#10335)
Should be calling Embeddings.embed_query instead of embed_documents when
searching
1 year ago
Bagatur f0ccce76fe
nuclia db nit (#10334) 1 year ago
Bagatur 205f406485
nuclia nb nit (#10331) 1 year ago