71 Commits (e6f0cee896592b25523f00a726f731538f704802)

Author	SHA1	Message	Date
Bagatur	0e4c5dd176	bump 13 (#10130)	1 year ago
maks-operlejn-ds	a8f804a618	Add data anonymizer (#9863) ... ### Description The feature for anonymizing data has been implemented. In order to protect private data, such as when querying external APIs (OpenAI), it is worth pseudonymizing sensitive data to maintain full privacy. Anonynization consists of two steps: 1. **Identification:** Identify all data fields that contain personally identifiable information (PII). 2. **Replacement**: Replace all PIIs with pseudo values or codes that do not reveal any personal information about the individual but can be used for reference. We're not using regular encryption, because the language model won't be able to understand the meaning or context of the encrypted data. We use *Microsoft Presidio* together with *Faker* framework for anonymization purposes because of the wide range of functionalities they provide. The full implementation is available in `PresidioAnonymizer`. ### Future works - **deanonymization** - add the ability to reverse anonymization. For example, the workflow could look like this: `anonymize -> LLMChain -> deanonymize`. By doing this, we will retain anonymity in requests to, for example, OpenAI, and then be able restore the original data. - **instance anonymization** - at this point, each occurrence of PII is treated as a separate entity and separately anonymized. Therefore, two occurrences of the name John Doe in the text will be changed to two different names. It is therefore worth introducing support for full instance detection, so that repeated occurrences are treated as a single object. ### Twitter handle @deepsense_ai / @MaksOpp --------- Co-authored-by: MaksOpp <maks.operlejn@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	d6957921f0	bump 276 (#9931)	1 year ago
Bagatur	9731ce5a40	bump 273 (#9751)	1 year ago
Predrag Gruevski	eee0d1d0dd	Update repository links in the package metadata. (#9454)	1 year ago
Bagatur	a69d1b84f4	bump 267 (#9403)	1 year ago
Bagatur	5935767056	bump lc 246, lce 9 (#9207)	1 year ago
Harrison Chase	4d526c49ed	bump experimental to 008 (#8490)	1 year ago
Harrison Chase	2448043b84	bump and fix (#8441)	1 year ago
Bagatur	61dd92f821	bump 246 (#8410)	1 year ago
Harrison Chase	ae78ef7fe6	bump experimental to 005 (#8339)	1 year ago
Bagatur	5c6dcb1960	bump 243 (#8289)	1 year ago
Bagatur	82b8d8596c	bump lc241 exp3 (#8193)	1 year ago
Bagatur	4928f7a9f5	undo bump (#8192)	1 year ago
Bagatur	d5689d58ab	Bagatur/bump 241 (#8182)	1 year ago
Harrison Chase	77bf75c236	bump experimental to 002 (#8150)	1 year ago
Harrison Chase	9f3073d418	bump versions (#8129)	1 year ago
Harrison Chase	aa0e69bc98	Harrison/official pre release (#8106)	1 year ago
Harrison Chase	8dcabd9205	bump releases rc0 (#8097)	1 year ago
Harrison Chase	d353d668e4	remove CVEs (#8092) ... This PR aims to move all code with CVEs into `langchain.experimental`. Note that we are NOT yet removing from the core `langchain` package - we will give people a week to migrate here. See MIGRATE.md for how to migrate Zero changes to functionality Vulnerabilities this addresses: PALChain: - https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5752409 - https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5759265 SQLDatabaseChain - https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5759268 `load_prompt` (Python files only) - https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5725807	1 year ago
Harrison Chase	da04760de1	Harrison/move experimental (#8084)	1 year ago