Commit Graph

2482 Commits (92b07ecaf32d4eff68ff80650ce57f7c4009f519)

Author SHA1 Message Date
Manuel Riezebosch 92b07ecaf3
DOCS: fix link to question answering (#13806)
first link in
[overview](https://python.langchain.com/docs/use_cases/question_answering/code_understanding#overview)
10 months ago
Chengzu Ou 4b8e053fe8
FEATURE: Add Databricks Vector Search as a new vector store (#13621)
**Description:**
This PR adds Databricks Vector Search as a new vector store in
LangChain.

- [x] Add `DatabricksVectorSearch` in `langchain/vectorstores/`
- [x] Unit tests
- [x] Add
[`databricks-vectorsearch`](https://pypi.org/project/databricks-vectorsearch/)
as a new optional dependency

We ran the following checks:
- `make format` passed  
- `make lint` failed but the failures were caused by other files
    + Files touched by this PR passed the linter  
- `make test` passed  
- `make coverage` failed but the failures were caused by other files.
Tests added by or related to this PR all passed
+ langchain/vectorstores/databricks_vector_search.py test coverage 94% 
- `make spell_check` passed  

The example notebook and updates to the [provider's documentation
page](https://github.com/langchain-ai/langchain/blob/master/docs/docs/integrations/providers/databricks.md)
will be added later in a separate PR.

**Dependencies:**
Optional dependency:
[`databricks-vectorsearch`](https://pypi.org/project/databricks-vectorsearch/)

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
10 months ago
Dylan Williams 1983a39894
FEATURE: Add OneNote document loader (#13841)
- **Description:** Added OneNote document loader
  - **Issue:** #12125
  - **Dependencies:** msal

Co-authored-by: Bagatur <baskaryan@gmail.com>
10 months ago
Ikko Eltociear Ashimine ff7d4d9c0b
Update llamacpp.ipynb (#13840)
specifed -> specified
10 months ago
Sᴜᴘᴇʀ Lᴇᴇ e42e95cc11
docs: fix link to `local_retrieval_qa` (#13872)
\The original link in [this
section](https://python.langchain.com/docs/use_cases/question_answering/#:~:text=locally%2Drunning%20models-,here,-.):

https://python.langchain.com/docs/modules/use_cases/question_answering/local_retrieval_qa

After fix:

https://python.langchain.com/docs/use_cases/question_answering/local_retrieval_qa
10 months ago
Yusuf Khan 935f78c944
FEATURE: Add retriever for Outline (#13889)
- **Description:** Added a retriever for the Outline API to ask
questions on knowledge base
  - **Issue:** resolves #11814
  - **Dependencies:** None
  - **Tag maintainer:** @baskaryan
10 months ago
ggeutzzang f2af82058f
DOCS: Fix Sample Code for Compatibility with Pydantic 2.0 (#13890)
- **Description:** 
I encountered an issue while running the existing sample code on the
page https://python.langchain.com/docs/modules/agents/how_to/agent_iter
in an environment with Pydantic 2.0 installed. The following error was
triggered:

```python
ValidationError                           Traceback (most recent call last)
<ipython-input-12-2ffff2c87e76> in <cell line: 43>()
     41 
     42 tools = [
---> 43     Tool(
     44         name="GetPrime",
     45         func=get_prime,

2 frames
/usr/local/lib/python3.10/dist-packages/pydantic/v1/main.py in __init__(__pydantic_self__, **data)
    339         values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
    340         if validation_error:
--> 341             raise validation_error
    342         try:
    343             object_setattr(__pydantic_self__, '__dict__', values)

ValidationError: 1 validation error for Tool
args_schema
  subclass of BaseModel expected (type=type_error.subclass; expected_class=BaseModel)
```

I have made modifications to the example code to ensure it functions
correctly in environments with Pydantic 2.0.
10 months ago
Stefano Lottini 19c68c7652
FEATURE: Astra DB, LLM cache classes (exact-match and semantic cache) (#13834)
This PR provides idiomatic implementations for the exact-match and the
semantic LLM caches using Astra DB as backend through the database's
HTTP JSON API. These caches require the `astrapy` library as dependency.

Comes with integration tests and example usage in the `llm_cache.ipynb`
in the docs.

@baskaryan this is the Astra DB counterpart for the Cassandra classes
you merged some time ago, tagging you for your familiarity with the
topic. Thank you!

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
10 months ago
Stefano Lottini 272df9dcae
Astra DB, chat message history (#13836)
This PR adds a chat message history component that uses Astra DB for
persistence through the JSON API.
The `astrapy` package is required for this class to work.

I have added tests and a small notebook, and updated the relevant
references in the other docs pages.

(@rlancemartin this is the counterpart of the Cassandra equivalent class
you so helpfully reviewed back at the end of June)

Thank you!
10 months ago
Bagatur 23566cbea9
DOCS: core editable dep api refs (#13747) 10 months ago
Bagatur 3d28c1a9e0
DOCS: fix core api ref build (#13744) 10 months ago
Harrison Chase d82cbf5e76
Separate out langchain_core package (#13577)
Co-authored-by: Nuno Campos <nuno@boringbits.io>
Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Erick Friis <erick@langchain.dev>
10 months ago
Bagatur 4eec47b191
DOCS: update rag use case images (#13615) 10 months ago
Sijun He 674bd90a47
DOCS: Fix typo in MongoDB memory docs (#13588)
- **Description:** Fix typo in MongoDB memory docs
  - **Tag maintainer:** @eyurtsev

<!-- Thank you for contributing to LangChain!

  - **Description:** Fix typo in MongoDB memory docs
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** @baskaryan
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
10 months ago
jwbeck97 a93616e972
FEAT: Add azure cognitive health tool (#13448)
- **Description:** This change adds an agent to the Azure Cognitive
Services toolkit for identifying healthcare entities
  - **Dependencies:** azure-ai-textanalytics (Optional)

---------

Co-authored-by: James Beck <James.Beck@sa.gov.au>
Co-authored-by: Bagatur <baskaryan@gmail.com>
10 months ago
John Mai 16f7912e1b
BUG: fix hunyuan appid type (#13496)
- **Description: fix hunyuan appid type
- **Issue:
https://github.com/langchain-ai/langchain/pull/12022#issuecomment-1815627855
10 months ago
Leonid Ganeline 43972be632
docs updating `AzureML` notebooks (#13492)
- Added/updated descriptions and links

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
10 months ago
Taranjeet Singh 47451764a7
Add embedchain retriever (#13553)
**Description:**

This commit adds embedchain retriever along with tests and docs.
Embedchain is a RAG framework to create data pipelines.

**Twitter handle:**
- [Taranjeet's twitter](https://twitter.com/taranjeetio) and
[Embedchain's twitter](https://twitter.com/embedchain)

**Reviewer**
@hwchase17

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
10 months ago
rafly lesmana 420a17542d
fix: Make YoutubeLoader support on demand language translation (#13583)
**Description:**
Enhance the functionality of YoutubeLoader to enable the translation of
available transcripts by refining the existing logic.

**Issue:**
Encountering a problem with YoutubeLoader (#13523) where the translation
feature is not functioning as expected.

Tag maintainers/contributors who might be interested:
@eyurtsev

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
10 months ago
Leonid Ganeline cc50e023d1
DOCS `langchain decorators` update (#13535)
added disclaimer

---------

Co-authored-by: Erick Friis <erickfriis@gmail.com>
10 months ago
Brace Sproul 02a13030c0
DOCS: updated langchain stack img to be svg (#13540) 10 months ago
Martin Krasser 79ed66f870
EXPERIMENTAL Generic LLM wrapper to support chat model interface with configurable chat prompt format (#8295)
## Update 2023-09-08

This PR now supports further models in addition to Lllama-2 chat models.
See [this comment](#issuecomment-1668988543) for further details. The
title of this PR has been updated accordingly.

## Original PR description

This PR adds a generic `Llama2Chat` model, a wrapper for LLMs able to
serve Llama-2 chat models (like `LlamaCPP`,
`HuggingFaceTextGenInference`, ...). It implements `BaseChatModel`,
converts a list of chat messages into the [required Llama-2 chat prompt
format](https://huggingface.co/blog/llama2#how-to-prompt-llama-2) and
forwards the formatted prompt as `str` to the wrapped `LLM`. Usage
example:

```python
# uses a locally hosted Llama2 chat model
llm = HuggingFaceTextGenInference(
    inference_server_url="http://127.0.0.1:8080/",
    max_new_tokens=512,
    top_k=50,
    temperature=0.1,
    repetition_penalty=1.03,
)

# Wrap llm to support Llama2 chat prompt format.
# Resulting model is a chat model
model = Llama2Chat(llm=llm)

messages = [
    SystemMessage(content="You are a helpful assistant."),
    MessagesPlaceholder(variable_name="chat_history"),
    HumanMessagePromptTemplate.from_template("{text}"),
]

prompt = ChatPromptTemplate.from_messages(messages)
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
chain = LLMChain(llm=model, prompt=prompt, memory=memory)

# use chat model in a conversation
# ...
```

Also part of this PR are tests and a demo notebook.

- Tag maintainer: @hwchase17
- Twitter handle: `@mrt1nz`

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
10 months ago
Bagatur 2e2114d2d0
FEATURE: Runnable with message history (#13418)
Add RunnableWithMessageHistory class that can wrap certain runnables and manages chat history for them.
10 months ago
Bagatur 0fc3af8932
IMPROVEMENT: update assistants output and doc (#13480) 10 months ago
Leonid Ganeline 21552628c8
DOCS updated `data_connection` index page (#13426)
- the `Index` section was missed. Created it.
- text simplification

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
10 months ago
Leonid Ganeline e3a5cd7969
docs `integrations/vectorstores/` cleanup (#13487)
- updated titles to consistent format
- added/updated descriptions and links
- format heading
10 months ago
Leonid Ganeline 1d2981114f
DOCS updated `async-faiss` example (#13434)
The original notebook has the `faiss` title which is duplicated in
the`faiss.jpynb`. As a result, we have two `faiss` items in the
vectorstore ToC. And the first item breaks the searching order (it is
placed between `A...` items).
- I updated title to `Asynchronous Faiss`.
10 months ago
Leonid Ganeline 9ff8f69e75
DOCS updated `memory` Titles (#13435)
- Fixed titles for two notebooks. They were inconsistent with other
titles and clogged ToC.
- Added `Upstash` description and link
- Moved the authentication text up in the `Elasticsearch` nb, right
after package installation. It was on the end of the page which was a
wrong place.
10 months ago
Stefano Lottini b029d9f4e6
Astra DB: minor improvements to docstrings and demo notebook (#13449)
This PR brings a few minor improvements to the docs, namely class/method
docstrings and the demo notebook.

- A note on how to control concurrency levels to tune performance in
bulk inserts, both in the class docstring and the demo notebook;
- Slightly increased concurrency defaults after careful experimentation
(still on the conservative side even for clients running on
less-than-typical network/hardware specs)
- renamed the DB token variable to the standardized
`ASTRA_DB_APPLICATION_TOKEN` name (used elsewhere, e.g. in the Astra DB
docs)
- added a note and a reference (add_text docstring, demo notebook) on
allowed metadata field names.

Thank you!
10 months ago
Leonid Ganeline 283ef1f66d
DOCS fix for `integratons/document_loaders` sidebar (#13471)
The current `integrations/document_loaders/` sidebar has the
`example_data` item, which is a menu with a single item: "Notebook".
It is happening because the `integrations/document_loaders/` folder has
the `example_data/notebook.md` file that is used to autogenerate the
above menu item.
- removed an example_data/notebook.md file. Docusaurus doesn't have
simple ways to fix this problem (to exclude folders/files from an
autogenerated sidebar). Removing this file didn't break any existing
examples, so this fix is safe.
10 months ago
Leonid Ganeline b1fcf5b481
DOCS: `integrations/text_embeddings/` cleanup (#13476)
Updated several notebooks:
- fixed titles which are inconsistent or break the ToC sorting order.
- added missed soruce descriptions and links
- fixed formatting
10 months ago
Bagatur 10fddac4b5
Bagatur/chain of note template(#13470) 10 months ago
Leonid Ganeline d5b1a21ae4
DOCS updated `semadb` example (#13431)
- the `SemaDB` notebook was placed in additional subfolder which breaks
the vectorstore ToC. I moved file up, removed this unnecessary
subfolder; updated the `vercel.json` with rerouting for the new URL
- Added SemaDB description and link
- improved text consistency
10 months ago
Leonid Ganeline 17c2007e0c
DOCS updated `Activeloop DeepMemory` notebook (#13428)
- Fixed the title of the notebook. It created an ugly ToC element as
`Activeloop DeepLake's DeepMemory + LangChain + ragas or how to get +27%
on RAG recall.`
- Added Activeloop description
- improved consistency in text
- fixed ToC (it was using HTML tagas that break left-side in-page ToC).
Now in-page ToC works
10 months ago
Bagatur 9e6748e198
DOCS: rag nit (#13436) 10 months ago
Leonid Ganeline 8a52c1456b
updated `clickup` example (#13424)
- Fixed headers (was more then 1 Titles)
- Removed security token value. It was OK to have it, because it is
temporary token, but the automatic security swippers raise warnings on
that.
- Added `ClickUp` service description and link.
10 months ago
Brace Sproul 79fa9a81f4
Fix a link in docs (#13423) 10 months ago
Bagatur f0bb839506
DOCS: langchain stack img update (#13421) 10 months ago
Bagatur 76c317ed78
DOCS: update rag use case (#13319) 10 months ago
Bagatur a0b39a4325
DOCS: install nit (#13380) 10 months ago
Bagatur 9f543634e2
Agent window management how to (#13033) 10 months ago
Leonid Ganeline c9b9359647
FEAT docs integration cards site (#13379)
The `Integrations` site is hidden now.
I've added it into the `More` menu.
The name is `Integration Cards` otherwise, it is confused with the
`Integrations` menu.

---------

Co-authored-by: Erick Friis <erickfriis@gmail.com>
10 months ago
Erick Friis 0f25ea9671
api doc newlines (#13378)
cc @leo-gan 

Deploying at
https://api.python.langchain.com/en/erick-api-doc-newlines-/api_reference.html
(will take a bit)
10 months ago
Leonid Ganeline 342ed5c77a
`Yi` model from `01.ai` , example (#13375)
Added an example with new soa `Yi` model to `HuggingFace-hub` notebook
10 months ago
Bagatur 3596be5210
DOCS: format notebooks (#13371) 10 months ago
Predrag Gruevski 2ebd167dba
Lint Python notebooks with ruff. (#12677)
The new ruff version fixed the blocking bugs, and I was able to fairly
easily us to a passing state: ruff fixed some issues on its own, I fixed
a handful by hand, and I added a list of narrowly-targeted exclusions
for files that are currently failing ruff rules that we probably should
look into eventually.

I went pretty lenient on the docs / cookbooks rules, allowing dead code
and such things. Perhaps in the future we may want to tighten the rules
further, but this is already a good set of checks that found real issues
and will prevent them going forward.
10 months ago
Leonid Ganeline f5bf3bdf14
added `Cookbooks` link (#13078)
It is a temporary solution before major documents refactoring.
Related to #13070 (not solving it)
10 months ago
Bagatur 1c67db4c18
Move OAI assistants to langchain and add callbacks (#13236) 10 months ago
Bagatur 8006919e52
DOCS: cleanup docs directory (#13301) 10 months ago
mertkayhan 9b4974871d
IMPROVEMENT Increase flexibility of ElasticVectorSearch (#6863)
Hey @rlancemartin, @eyurtsev ,

I did some minimal changes to the `ElasticVectorSearch` client so that
it plays better with existing ES indices.

Main changes are as follows:

1. You can pass the dense vector field name into `_default_script_query`
2. You can pass a custom script query implementation and the respective
parameters to `similarity_search_with_score`
3. You can pass functions for building page content and metadata for the
resulting `Document`

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  4. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @dev2049
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @dev2049
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @vowelparrot
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
10 months ago