Commit Graph

64 Commits (d0b818b634ff72ad6e50205abebeb7e822b21c14)

Author SHA1 Message Date
Bagatur a20e8f8bb0
experimental[patch]: release 0.0.43 (#13570) 10 months ago
Bagatur 48fbc5513d
infra[patch], langchain[patch]: fix test deps and upper bound langchain dep on core(#13984) 10 months ago
Nuno Campos 8329f81072
Use pytest asyncio auto mode (#13643)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
11 months ago
Bagatur 78a1f4b264
bump 338, exp 42 (#13564) 11 months ago
Martin Krasser 79ed66f870
EXPERIMENTAL Generic LLM wrapper to support chat model interface with configurable chat prompt format (#8295)
## Update 2023-09-08

This PR now supports further models in addition to Lllama-2 chat models.
See [this comment](#issuecomment-1668988543) for further details. The
title of this PR has been updated accordingly.

## Original PR description

This PR adds a generic `Llama2Chat` model, a wrapper for LLMs able to
serve Llama-2 chat models (like `LlamaCPP`,
`HuggingFaceTextGenInference`, ...). It implements `BaseChatModel`,
converts a list of chat messages into the [required Llama-2 chat prompt
format](https://huggingface.co/blog/llama2#how-to-prompt-llama-2) and
forwards the formatted prompt as `str` to the wrapped `LLM`. Usage
example:

```python
# uses a locally hosted Llama2 chat model
llm = HuggingFaceTextGenInference(
    inference_server_url="http://127.0.0.1:8080/",
    max_new_tokens=512,
    top_k=50,
    temperature=0.1,
    repetition_penalty=1.03,
)

# Wrap llm to support Llama2 chat prompt format.
# Resulting model is a chat model
model = Llama2Chat(llm=llm)

messages = [
    SystemMessage(content="You are a helpful assistant."),
    MessagesPlaceholder(variable_name="chat_history"),
    HumanMessagePromptTemplate.from_template("{text}"),
]

prompt = ChatPromptTemplate.from_messages(messages)
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
chain = LLMChain(llm=model, prompt=prompt, memory=memory)

# use chat model in a conversation
# ...
```

Also part of this PR are tests and a demo notebook.

- Tag maintainer: @hwchase17
- Twitter handle: `@mrt1nz`

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
11 months ago
Bagatur a9b2c943e6
bump 336, exp 44 (#13420) 11 months ago
Predrag Gruevski d63d4994c0
Bump all libraries to the latest `ruff` version. (#13350)
This version of `ruff` is the one we'll be using to lint the docs and
cookbooks (#12677), so I'm making it used everywhere else too.
11 months ago
Bagatur 24386e0860
bump 334, exp 40 (#13211) 11 months ago
Bagatur 1f85ec34d5
bump 331rc3 exp 39 (#13086) 11 months ago
Bagatur cf481c9418
bump exp 38 (#13016) 11 months ago
Bagatur eee5181b7a
bump 328, exp 37 (#12722) 11 months ago
Predrag Gruevski f94e24dfd7
Install and use `ruff format` instead of black for code formatting. (#12585)
Best to review one commit at a time, since two of the commits are 100%
autogenerated changes from running `ruff format`:
- Install and use `ruff format` instead of black for code formatting.
- Output of `ruff format .` in the `langchain` package.
- Use `ruff format` in experimental package.
- Format changes in experimental package by `ruff format`.
- Manual formatting fixes to make `ruff .` pass.
11 months ago
Harrison Chase eb903e211c
bump to 36 (#12487) 11 months ago
Bagatur c6a733802b
bump 324 and 35 (#12352) 12 months ago
Bagatur 286a29a49e
bump 322 and 34 (#12228) 12 months ago
Bagatur 963ff93476
bump 321 (#12161) 12 months ago
Bagatur 85302a9ec1
Add CI check that integration tests compile (#12090) 12 months ago
Bagatur 35c7c1f050
bump 317 (#11986) 12 months ago
Predrag Gruevski dcd0392423
Upgrade to newer black (23.10) and ruff (first 0.1.x!) versions. (#11944)
Minor lint dependency version upgrade to pick up latest functionality.

Ruff's new v0.1 version comes with lots of nice features, like
fix-safety guarantees and a preview mode for not-yet-stable features:
https://astral.sh/blog/ruff-v0.1.0
12 months ago
Bagatur ba0d729961
bump 316 (#11928) 12 months ago
Bagatur 25b1d65305
bump 315 (#11850) 12 months ago
Erick Friis 1861cc7100
General anthropic functions, steps towards experimental integration tests (#11727)
To match change in js here
https://github.com/langchain-ai/langchainjs/pull/2892

Some integration tests need a bit more work in experimental:
![Screenshot 2023-10-12 at 12 02 49
PM](https://github.com/langchain-ai/langchain/assets/9557659/262d7d22-c405-40e9-afef-669e8d585307)

Pretty sure the sqldatabase ones are an actual regression or change in
interface because it's returning a placeholder.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
12 months ago
Bagatur 9c0584be74
bump 313 (#11718) 12 months ago
Bagatur 7232e082de
bump 312 (#11621) 1 year ago
Bagatur 53887242a1
bump 310 (#11486) 1 year ago
Bagatur 8fafa1af91 merge 1 year ago
Bagatur 8b6b8bf68c
bump 309 (#11443) 1 year ago
Predrag Gruevski c9986bc3a9
Tweak type hints to match dependency's behavior. (#11355)
Needs #11353 to merge first, and a new `langchain` to be published with
those changes.
1 year ago
Bagatur 16a80779b9
bump 307 (#11380) 1 year ago
Bagatur 8eec43ed91
bump 306 (#11289) 1 year ago
Bagatur 77c7c9ab97
bump 305 (#11224) 1 year ago
Bagatur 12fb393a43
bump 302 (#11070) 1 year ago
Bagatur aa6e6db8c7
bump 301 (#11018) 1 year ago
Bagatur 24cb5cd379
bump 298 (#10892) 1 year ago
Bagatur 46aa90062b
bump exp 19 (#10851) 1 year ago
Bagatur 0d1550da91
Bagatur/bump 295 (#10785) 1 year ago
Bagatur f7f3c02585
bump 287 (#10498) 1 year ago
olgavrou b78d672a43 merge from upstream/master 1 year ago
olgavrou 11f20cded1 move everything into experimental 1 year ago
Bagatur d2d11ccf63
bump 285 (#10373) 1 year ago
Bagatur 672907bbbb
bump 284 (#10330) 1 year ago
Tomaz Bratanic db73c9d5b5
Diffbot Graph Transformer / Neo4j Graph document ingestion (#9979)
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur 098b4aa465
bump 281 (#10189) 1 year ago
Bagatur 0e4c5dd176
bump 13 (#10130) 1 year ago
maks-operlejn-ds a8f804a618
Add data anonymizer (#9863)
### Description

The feature for anonymizing data has been implemented. In order to
protect private data, such as when querying external APIs (OpenAI), it
is worth pseudonymizing sensitive data to maintain full privacy.

Anonynization consists of two steps:

1. **Identification:** Identify all data fields that contain personally
identifiable information (PII).
2. **Replacement**: Replace all PIIs with pseudo values or codes that do
not reveal any personal information about the individual but can be used
for reference. We're not using regular encryption, because the language
model won't be able to understand the meaning or context of the
encrypted data.

We use *Microsoft Presidio* together with *Faker* framework for
anonymization purposes because of the wide range of functionalities they
provide. The full implementation is available in `PresidioAnonymizer`.

### Future works

- **deanonymization** - add the ability to reverse anonymization. For
example, the workflow could look like this: `anonymize -> LLMChain ->
deanonymize`. By doing this, we will retain anonymity in requests to,
for example, OpenAI, and then be able restore the original data.
- **instance anonymization** - at this point, each occurrence of PII is
treated as a separate entity and separately anonymized. Therefore, two
occurrences of the name John Doe in the text will be changed to two
different names. It is therefore worth introducing support for full
instance detection, so that repeated occurrences are treated as a single
object.

### Twitter handle
@deepsense_ai / @MaksOpp

---------

Co-authored-by: MaksOpp <maks.operlejn@gmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur d6957921f0
bump 276 (#9931) 1 year ago
Bagatur 9731ce5a40
bump 273 (#9751) 1 year ago
Predrag Gruevski eee0d1d0dd
Update repository links in the package metadata. (#9454) 1 year ago
Bagatur a69d1b84f4
bump 267 (#9403) 1 year ago
Bagatur 5935767056
bump lc 246, lce 9 (#9207) 1 year ago