Commit Graph

35 Commits (2f490be09b13ec223e03a9e9eaf2e53e65d77c85)

Author SHA1 Message Date
Predrag Gruevski c9986bc3a9
Tweak type hints to match dependency's behavior. (#11355)
Needs #11353 to merge first, and a new `langchain` to be published with
those changes.
12 months ago
Bagatur 16a80779b9
bump 307 (#11380) 12 months ago
Bagatur 8eec43ed91
bump 306 (#11289) 12 months ago
Bagatur 77c7c9ab97
bump 305 (#11224) 12 months ago
Bagatur 12fb393a43
bump 302 (#11070) 12 months ago
Bagatur aa6e6db8c7
bump 301 (#11018) 1 year ago
Bagatur 24cb5cd379
bump 298 (#10892) 1 year ago
Bagatur 46aa90062b
bump exp 19 (#10851) 1 year ago
Bagatur 0d1550da91
Bagatur/bump 295 (#10785) 1 year ago
Bagatur f7f3c02585
bump 287 (#10498) 1 year ago
Bagatur d2d11ccf63
bump 285 (#10373) 1 year ago
Bagatur 672907bbbb
bump 284 (#10330) 1 year ago
Tomaz Bratanic db73c9d5b5
Diffbot Graph Transformer / Neo4j Graph document ingestion (#9979)
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur 098b4aa465
bump 281 (#10189) 1 year ago
Bagatur 0e4c5dd176
bump 13 (#10130) 1 year ago
maks-operlejn-ds a8f804a618
Add data anonymizer (#9863)
### Description

The feature for anonymizing data has been implemented. In order to
protect private data, such as when querying external APIs (OpenAI), it
is worth pseudonymizing sensitive data to maintain full privacy.

Anonynization consists of two steps:

1. **Identification:** Identify all data fields that contain personally
identifiable information (PII).
2. **Replacement**: Replace all PIIs with pseudo values or codes that do
not reveal any personal information about the individual but can be used
for reference. We're not using regular encryption, because the language
model won't be able to understand the meaning or context of the
encrypted data.

We use *Microsoft Presidio* together with *Faker* framework for
anonymization purposes because of the wide range of functionalities they
provide. The full implementation is available in `PresidioAnonymizer`.

### Future works

- **deanonymization** - add the ability to reverse anonymization. For
example, the workflow could look like this: `anonymize -> LLMChain ->
deanonymize`. By doing this, we will retain anonymity in requests to,
for example, OpenAI, and then be able restore the original data.
- **instance anonymization** - at this point, each occurrence of PII is
treated as a separate entity and separately anonymized. Therefore, two
occurrences of the name John Doe in the text will be changed to two
different names. It is therefore worth introducing support for full
instance detection, so that repeated occurrences are treated as a single
object.

### Twitter handle
@deepsense_ai / @MaksOpp

---------

Co-authored-by: MaksOpp <maks.operlejn@gmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur d6957921f0
bump 276 (#9931) 1 year ago
Bagatur 9731ce5a40
bump 273 (#9751) 1 year ago
Predrag Gruevski eee0d1d0dd
Update repository links in the package metadata. (#9454) 1 year ago
Bagatur a69d1b84f4
bump 267 (#9403) 1 year ago
Bagatur 5935767056
bump lc 246, lce 9 (#9207) 1 year ago
Harrison Chase 4d526c49ed
bump experimental to 008 (#8490) 1 year ago
Harrison Chase 2448043b84
bump and fix (#8441) 1 year ago
Bagatur 61dd92f821
bump 246 (#8410) 1 year ago
Harrison Chase ae78ef7fe6
bump experimental to 005 (#8339) 1 year ago
Bagatur 5c6dcb1960
bump 243 (#8289) 1 year ago
Bagatur 82b8d8596c
bump lc241 exp3 (#8193) 1 year ago
Bagatur 4928f7a9f5
undo bump (#8192) 1 year ago
Bagatur d5689d58ab
Bagatur/bump 241 (#8182) 1 year ago
Harrison Chase 77bf75c236
bump experimental to 002 (#8150) 1 year ago
Harrison Chase 9f3073d418
bump versions (#8129) 1 year ago
Harrison Chase aa0e69bc98
Harrison/official pre release (#8106) 1 year ago
Harrison Chase 8dcabd9205
bump releases rc0 (#8097) 1 year ago
Harrison Chase d353d668e4
remove CVEs (#8092)
This PR aims to move all code with CVEs into `langchain.experimental`.
Note that we are NOT yet removing from the core `langchain` package - we
will give people a week to migrate here.

See MIGRATE.md for how to migrate

Zero changes to functionality

Vulnerabilities this addresses:

PALChain:
- https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5752409
- https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5759265

SQLDatabaseChain
- https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5759268

`load_prompt` (Python files only)
- https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5725807
1 year ago
Harrison Chase da04760de1
Harrison/move experimental (#8084) 1 year ago