Commit Graph

4319 Commits (fed137a8a9a1387278d02b4533016a11ae631873)
 

Author SHA1 Message Date
skspark 52a3e8a261
Add integration TCs on bing search (#8068) (#10021)
## Description
Added integration TCs on bing search utility

## Issue
#8068 

## Dependencies
None
1 year ago
Hyeokjun seo e2e05ad89e
Fix Typo : `openai_api_key` -> `serpapi_api_key` (#10020)
Fixed typo in the comments Notebook. (which says `openai_api_key` for
SerpAPI)
1 year ago
Tomaz Bratanic f2e8399cc8
Fix link in Neo4j provider page (#10023) 1 year ago
William FH 5341b04d68
Update error message (#9970)
in evals
1 year ago
William FH b82ad19ed2
Check memory address (#9971)
Don't want to dup the collector but can have multiple
1 year ago
Bagatur e805f8e263 add tests 1 year ago
Bagatur 1f5c579ef4 add 1 year ago
Bagatur 240cc289e6 wip 1 year ago
Bagatur 7fa82900cb
guides docs nits (#10005) 1 year ago
Bagatur 2f03e71e67
rename local llm guide (#10004) 1 year ago
Bagatur 781f274d19
make privacy guide section (#10003) 1 year ago
maks-operlejn-ds a8f804a618
Add data anonymizer (#9863)
### Description

The feature for anonymizing data has been implemented. In order to
protect private data, such as when querying external APIs (OpenAI), it
is worth pseudonymizing sensitive data to maintain full privacy.

Anonynization consists of two steps:

1. **Identification:** Identify all data fields that contain personally
identifiable information (PII).
2. **Replacement**: Replace all PIIs with pseudo values or codes that do
not reveal any personal information about the individual but can be used
for reference. We're not using regular encryption, because the language
model won't be able to understand the meaning or context of the
encrypted data.

We use *Microsoft Presidio* together with *Faker* framework for
anonymization purposes because of the wide range of functionalities they
provide. The full implementation is available in `PresidioAnonymizer`.

### Future works

- **deanonymization** - add the ability to reverse anonymization. For
example, the workflow could look like this: `anonymize -> LLMChain ->
deanonymize`. By doing this, we will retain anonymity in requests to,
for example, OpenAI, and then be able restore the original data.
- **instance anonymization** - at this point, each occurrence of PII is
treated as a separate entity and separately anonymized. Therefore, two
occurrences of the name John Doe in the text will be changed to two
different names. It is therefore worth introducing support for full
instance detection, so that repeated occurrences are treated as a single
object.

### Twitter handle
@deepsense_ai / @MaksOpp

---------

Co-authored-by: MaksOpp <maks.operlejn@gmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur 98cce7dcd3
update moderation docs (#10002) 1 year ago
Bagatur b3e3a31240
bump 277 (#9997) 1 year ago
Bagatur 9828701de1
mv base cache to schema (#9953)
if you remove all other imports from langchain.init it exposes a
circular dep
1 year ago
Christophe Bornet 9870bfb9cd
Add bucket and object key to metadata in S3 loader (#9317)
- Description: this PR adds `s3_object_key` and `s3_bucket` to the doc
metadata when loading an S3 file. This is particularly useful when using
`S3DirectoryLoader` to remove the files from the dir once they have been
processed (getting the object keys from the metadata `source` field
seems brittle)
  - Dependencies: N/A
  - Tag maintainer: ?
  - Twitter handle: _cbornet

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
1 year ago
Eugene Yurtsev 6da158388b Merge branch 'master' into ywkim/master 1 year ago
Guy Korland 24c0b01c38
Extend the FalkorDB QA demo (#9992)
- Description: Extend the FalkorDB QA demo
  - Tag maintainer: @baskaryan
1 year ago
Eugene Yurtsev 588237ef30
Make document serializable, create utility to create a docstore (#9674)
This PR makes the following changes:

1. Documents become serializable using langhchain serialization
2. Make a utility to create a docstore kw store

Will help to address issue here:
https://github.com/langchain-ai/langchain/issues/9345
1 year ago
Eugene Yurtsev e8f29be350 x 1 year ago
Buckler89 a28e888b36
fix call _get_keys for custom_evaluator (#9763)
In the function _load_run_evaluators the function _get_keys was not
called if only custom_evaluators parameter is used


- Description: In the function _load_run_evaluators the function
_get_keys was not called if only custom_evaluators parameter is used,
  - Issue: no issue created for this yet,
  - Dependencies: None,
  - Tag maintainer: @vowelparrot,
  - Twitter handle: Buckler89

---------

Co-authored-by: ddroghini <d.droghini@mflgroup.com>
1 year ago
Eugene Yurtsev cafce9ed23 x 1 year ago
wlleiiwang 8c4e29240c implement vectorstores by tencent vectordb 1 year ago
Bagatur 2d2b097fab
mv chat history (#9725) 1 year ago
Bagatur d762a6b51f
rm mutable defaults (#9974) 1 year ago
Arjun Aravindan 6a51672164
Update SeleniumURLLoader to use webdriver Service in favor of deprecated executable_path parameter (#9814)
Description: This commit uses the new Service object in Selenium
webdriver as executable_path has been [deprecated and removed in
selenium version
4.11.2](9f5801c82f)
Issue: https://github.com/langchain-ai/langchain/issues/9808
Tag Maintainer: @eyurtsev
1 year ago
William FH c844aaa7a6
Weakref to tracer (#9954)
Prevent memory/thread leakage
1 year ago
Jurik-001 a05fed9369
Fix add callbacks to spark_sql due to depreciation of callback_manager (#9831)
Description: Due to depreciation (regarding to line 109 in
[langchain/libs/langchain/langchain/chains/base.py](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/chains/base.py)
of callback_manager i replaced several parts

Issue: None
Dependencies: 
Maintainer: @baskaryan

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
dafu c26deb6b38
fixed openai_functions api_response format args err
root cause: args may not have a key (params) resulting in an error
1 year ago
axiangcoding ffa5625134
feat(llms): improve ERNIE-Bot chat model (#9833)
- Description: improve ERNIE-Bot chat model, add request timeout and
more testcases.
  - Issue: None
  - Dependencies: None
  - Tag maintainer: @baskaryan

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur bdccb1215a
docs: `integrations/tools` consistency (#9965)
Updated titles, descriptions into consistent format.
1 year ago
Bagatur d966ba63e2
fixed GoogleCloudEnterpriseSearchRetriever returning an empty array (#9858)
`GoogleCloudEnterpriseSearchRetriever` returned an empty array of
documents earlier, fixed
1 year ago
Bagatur ec362ecbe2
Fixed regex bug in RetrievalQAWithSources in previous update (#9898)
- Description: In my previous PR, I had modified the code to catch all
kinds of [SOURCES, sources, Source, Sources]. However, this change
included checking for a colon or a white space which should actually
have been only checking for a colon.
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
1 year ago
Nikhil Suresh 56a0165a4e cleaned up unit test example 1 year ago
William FH cedfad541d
don't emit none from eval config (#9963) 1 year ago
Nikhil Suresh b31475c622 minor updates to regex 1 year ago
Leonid Ganeline d03d6f6fd9
Merge branch 'master' into docs-tools-menu 1 year ago
Bagatur 8fb0a9594c
Add LLMonitor Callback Handler Integration - open-source observability & analytics (#9870)
Adds support for [llmonitor](https://llmonitor.com) callbacks.

It enables:
- Requests tracking / logging / analytics
- Error debugging
- Cost analytics
- User tracking

Let me know if anythings neds to be changed for merge.

Thank you!
1 year ago
Bagatur 4eeba88905
Use unified Python setup steps for release workflow. (#9861)
Using the same Python setup GitHub Action step as the lint and test
workflows.
1 year ago
leo-gan 8c1678a8c7 Updated titles, descriptions. 1 year ago
William FH d799963870
Wfh/async tool (#9878)
Co-authored-by: Daniel Brenot <dbrenot@pelmorex.com>
Co-authored-by: Daniel <daniel.alexander.brenot@gmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur 7bba1d911b
Fix typo in code_understanding.ipynb (#9899)
seperate -> separate
1 year ago
Bagatur 2e65434568
docs: Fix the syntax error, replace "dotenv.load_env()" with "dotenv.… (#9900)
Description: The documents incorrectly mentions "dotenv.load_env()", but
it should actually be "dotenv.load_dotenv()". You can see the screenshot
below for reference:

python-dotenv: 1.0.0


![image](https://github.com/langchain-ai/langchain/assets/2959046/94dc4b51-cc2f-412d-92e9-16b8ff0d513e)
1 year ago
Bagatur b416f5c0c8
fix a link name format to the dependents document (#9928) 1 year ago
Bagatur 8f199239b8
docs: `llms/google vertex AI` example update (#9960)
Updated title, description, added sections.
1 year ago
Bagatur 2a03a0087d
docs: `memory` menu (#9947)
The [Memory](https://python.langchain.com/docs/modules/memory/) menu is
clogged with unnecessary wording.
I've made it more concise by simplifying titles of the example
notebooks.
As results, menu is shorter and better for comprehend.
1 year ago
Bagatur f7cc125cac
docs: `memory types` menu (#9949)
The [Memory
Types](https://python.langchain.com/docs/modules/memory/types/) menu is
clogged with unnecessary wording.
I've made it more concise by simplifying titles of the example
notebooks.
As results, menu is shorter and better for comprehend.
1 year ago
Bagatur 16eb935469
Fix for similarity_search_with_score (#9903)
- Description: the implementation for similarity_search_with_score did
not actually include a score or logic to filter. Now fixed.
- Tag maintainer: @rlancemartin
- Twitter handle: @ofermend
1 year ago
Bagatur c70bb0ec28
Activeloopai runtime arg (#9961) 1 year ago
Bagatur 0f85671630 fmt 1 year ago