Commit Graph

5051 Commits

Author SHA1 Message Date
Attila Tőkés
ba9371854f
OpenAI gpt-3.5-turbo-instruct cost information (#11218)
Added pricing info for `gpt-3.5-turbo-instruct` for OpenAI and Azure
OpenAI.

Co-authored-by: Attila Tőkés <atokes@rws.com>
2023-09-29 08:44:55 -07:00
Eugene Yurtsev
de69ea26e8
Suppress warnings in interactive env that stem from tab completion (#11190)
Suppress warnings in interactive environments that can arise from users 
relying on tab completion (without even using deprecated modules).

jupyter seems to filter warnings by default (at least for me), but
ipython surfaces them all
2023-09-29 11:44:30 -04:00
Jon Saginaw
715ffda28b
mongodb doc loader init (#10645)
- **Description:** A Document Loader for MongoDB
  - **Issue:** n/a
  - **Dependencies:** Motor, the async driver for MongoDB
  - **Tag maintainer:** n/a
  - **Twitter handle:** pigpenblue

Note that an initial mongodb document loader was created 4 months ago,
but the [PR ](https://github.com/langchain-ai/langchain/pull/4285)was
never pulled in. @leo-gan had commented on that PR, but given it is
extremely far behind the master branch and a ton has changed in
Langchain since then (including repo name and structure), I rewrote the
branch and issued a new PR with the expectation that the old one can be
closed.

Please reference that old PR for comments/context, but it can be closed
in favor of this one. Thanks!

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2023-09-29 11:44:07 -04:00
Cynthia Yang
523898ab9c
Update fireworks features (#11205)
Description
* Update fireworks feature on web page

Issue - Not applicable
Dependencies - None
Tag maintainer - @baskaryan
2023-09-29 08:37:06 -07:00
Nuno Campos
3d8aa88e26 Add async tests and comments 2023-09-29 15:28:46 +01:00
Nuno Campos
4ad0f3de2b
Add RunnableGenerator (#11214)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
2023-09-29 15:21:37 +01:00
Guy Korland
748a757306
Clean warnings: replace type with isinstance and fix syntax (#11219)
Clean warnings: replace type with `isinstance` and fix on notebook
syntax syntax
2023-09-29 10:06:33 -04:00
Nuno Campos
091d8845d5 Backwards compat 2023-09-29 14:18:38 +01:00
Nuno Campos
4e28a7a513 Implement diff 2023-09-29 14:12:48 +01:00
Nuno Campos
5cbe2b7b6a Implement diff 2023-09-29 14:12:18 +01:00
Nuno Campos
6c0a6b70e0 WIP Add tests§ 2023-09-29 14:11:34 +01:00
Nuno Campos
63f2ef8d1c Implement str one 2023-09-29 14:11:34 +01:00
Nuno Campos
f672b39cc9 Add a streaming json parser 2023-09-29 14:11:34 +01:00
Nuno Campos
2387647d30 Lint 2023-09-29 14:11:03 +01:00
Nuno Campos
0318cdd33c Add tests 2023-09-29 12:25:19 +01:00
Nuno Campos
b67db8deaa Add RunnableGenerator 2023-09-29 12:04:32 +01:00
Nuno Campos
ca5293bf54
Enable creating Tools from any Runnable (#11177)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
2023-09-29 12:03:56 +01:00
Nuno Campos
e35ea565d1 Lint 2023-09-29 12:00:56 +01:00
Nuno Campos
7f589ebbc2 Lint 2023-09-29 11:57:01 +01:00
Nuno Campos
8be598f504 Fix invocation 2023-09-29 11:57:01 +01:00
Nuno Campos
6eb6c45c98 Enable creating Tools from any Runnable 2023-09-29 11:57:01 +01:00
Nuno Campos
61b5942adf
Implement better reprs for Runnables (#11175)
```
ChatPromptTemplate(messages=[SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], template='You are a nice assistant.')), HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['question'], template='{question}'))])
| RunnableLambda(lambda x: x)
| {
    chat: FakeListChatModel(responses=["i'm a chatbot"]),
    llm: FakeListLLM(responses=["i'm a textbot"])
  }
```

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
2023-09-29 11:56:28 +01:00
Nuno Campos
e8e2b812c9 Even more 2023-09-29 11:54:22 +01:00
Nuno Campos
fc072100fa skip more 2023-09-29 11:51:48 +01:00
Nuno Campos
7bfee012d5 Skip in py3.8 2023-09-29 11:49:12 +01:00
Nuno Campos
b8e3e1118d Skip for py3.8 2023-09-29 11:45:20 +01:00
William FH
db05ea2b78
Add from_embeddings for opensearch (#10957) 2023-09-29 00:00:58 -07:00
William FH
73693c18fc
Add support for project metadata in run_on_dataset (#11200) 2023-09-28 21:26:37 -07:00
James Braza
b11f21c25f
Updated LocalAIEmbeddings docstring to better explain why openai (#10946)
Fixes my misgivings in
https://github.com/langchain-ai/langchain/issues/10912
2023-09-28 19:56:42 -07:00
Eugene Yurtsev
2c114fcb5e
Fix web-base loader (#11135)
Fix initialization

https://github.com/langchain-ai/langchain/issues/11095
2023-09-28 19:36:46 -07:00
jreinjr
3bc44b01c0
Typo fix to MathpixPDFLoader - changed processed_file_format default … (#10960)
…from mmd to md. https://github.com/langchain-ai/langchain/issues/7282

<!-- 
- **Description:** minor fix to a breaking typo - MathPixPDFLoader
processed_file_format is "mmd" by default, doesn't work, changing to
"md" fixes the issue,
- **Issue:** 7282
(https://github.com/langchain-ai/langchain/issues/7282),
  - **Dependencies:** none,
  - **Tag maintainer:** @hwchase17,
  - **Twitter handle:** none
 -->

Co-authored-by: jare0530 <7915+jare0530@users.noreply.ghe.oculus-rep.com>
2023-09-28 19:03:30 -07:00
Dr. Fabien Tarrade
66415eed6e
Support new version of tiktoken that are working with langchain (tag "^0.3.2" => "">=0.3.2,<0.6.0" and python "^3.9" =>">=3.9") (#11006)
- **Description:**
be able to use langchain with other version than tiktoken 0.3.3 i.e
0.5.1
  - **Issue:**
cannot installed the conda-forge version since it applied all optional
dependency:
       https://github.com/conda-forge/langchain-feedstock/pull/85  
replace "^0.3.2" by "">=0.3.2,<0.6.0" and "^3.9" by python=">=3.9"
      Tested with python 3.10, langchain=0.0.288 and tiktoken==0.5.0

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-09-28 18:53:24 -07:00
Clément Sicard
1b48d6cb8c
LlamaCppEmbeddings: adds verbose parameter, similar to llms.LlamaCpp class (#11038)
## Description

As of now, when instantiating and during inference, `LlamaCppEmbeddings`
outputs (a lot of) verbose when controlled from Langchain binding - it
is a bit annoying when computing the embeddings of long documents, for
instance.

This PR adds `verbose` for `LlamaCppEmbeddings` objects to be able
**not** to print the verbose of the model to `stderr`. It is natively
supported by `llama-cpp-python` and directly passed to the library – the
PR is hence very small.

The value of `verbose` is `True` by default, following the way it is
defined in [`LlamaCpp` (`llamacpp.py`
#L136-L137)](c87e9fb2ce/libs/langchain/langchain/llms/llamacpp.py (L136-L137))

## Issue

_No issue linked_

## Dependencies

_No additional dependency needed_

## To see it in action

```python
from langchain.embeddings import LlamaCppEmbeddings

MODEL_PATH = "<path_to_gguf_file>"

if __name__ == "__main__":
    llm_embeddings = LlamaCppEmbeddings(
        model_path=MODEL_PATH,
        n_gpu_layers=1,
        n_batch=512,
        n_ctx=2048,
        f16_kv=True,
        verbose=False,
    )
```

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-09-28 18:37:51 -07:00
Noah Czelusta
a00a73ef18
Add last_edited_time and created_time props to NotionDBLoader (#11020)
# Description

Adds logic for NotionDBLoader to correctly populate `last_edited_time`
and `created_time` fields from [page
properties](https://developers.notion.com/reference/page#property-value-object).

There are no relevant tests for this code to be updated.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-09-28 18:37:34 -07:00
Eugene Yurtsev
e06e84b293
LangServe: Relax requirements (#11198)
Relax requirements
2023-09-28 21:27:19 -04:00
PaperMoose
5d7c6d1bca
Synthetic Data generation (#9472)
---------

Co-authored-by: William Fu-Hinthorn <13333726+hinthornw@users.noreply.github.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-09-28 18:16:05 -07:00
Donatas Remeika
a4e0cf6300
SearchApi integration (#11023)
Based on the customers' requests for native langchain integration,
SearchApi is ready to invest in AI and LLM space, especially in
open-source development.

- This is our initial PR and later we want to improve it based on
customers' and langchain users' feedback. Most likely changes will
affect how the final results string is being built.
- We are creating similar native integration in Python and JavaScript.
- The next plan is to integrate into Java, Ruby, Go, and others.
- Feel free to assign @SebastjanPrachovskij as a main reviewer for any
SearchApi-related searches. We will be glad to help and support
langchain development.
2023-09-28 18:08:37 -07:00
Bagatur
8cd18a48e4
fix trubrics lint issue (#11202) 2023-09-28 18:07:50 -07:00
Fynn Flügge
b738ccd91e
chore: add support for TypeScript code splitting (#11160)
- **Description:** Adds typescript language to `TextSplitter`

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-09-28 16:41:51 -07:00
Kenneth Choe
17fcbed92c
Support add_embeddings for opensearch (#11050)
- **Description:**
      -  Make running integration test for opensearch easy
- Provide a way to use different text for embedding: refer to #11002 for
more of the use case and design decision.
  - **Issue:** N/A
  - **Dependencies:** None other than the existing ones.
2023-09-28 16:41:11 -07:00
Jeff Kayne
c586f6dc1b
Callback integration for Trubrics (#11059)
After contributing to some examples in the
[langsmith-cookbook](https://github.com/langchain-ai/langsmith-cookbook)
with @hinthornw, here is a PR that adds a callback handler to use
LangChain with [Trubrics](https://github.com/trubrics/trubrics-sdk).
2023-09-28 16:20:19 -07:00
Michael Landis
a8db594012
fix: short-circuit black and mypy calls when no changes made (#11051)
Both black and mypy expect a list of files or directories as input.
As-is the Makefile computes a list files changed relative to the last
commit; these are passed to black and mypy in the `format_diff` and
`lint_diff` targets. This is done by way of the Makefile variable
`PYTHON_FILES`. This is to save time by skipping running mypy and black
over the whole source tree.

When no changes have been made, this variable is empty, so the call to
black (and mypy) lacks input files. The call exits with error causing
the Makefile target to error out with:

```bash
$ make format_diff
poetry run black
Usage: black [OPTIONS] SRC ...

One of 'SRC' or 'code' is required.
make: *** [format_diff] Error 1
```

This is unexpected and undesirable, as the naive caller (that's me! 😄 )
will think something else is wrong. This commit smooths over this by
short circuiting when `PYTHON_FILES` is empty.
2023-09-28 16:13:07 -07:00
Michael Kim
fbcd8e02f2
Change type annotations from LLMChain to Chain in MultiPromptChain (#11082)
- **Description:** The types of 'destination_chains' and 'default_chain'
in 'MultiPromptChain' were changed from 'LLMChain' to 'Chain'. and
removed variables declared overlapping with the parent class
- **Issue:** When a class that inherits only Chain and not LLMChain,
such as 'SequentialChain' or 'RetrievalQA', is entered in
'destination_chains' and 'default_chain', a pydantic validation error is
raised.
-  -  codes
```
retrieval_chain = ConversationalRetrievalChain(
        retriever=doc_retriever,
        combine_docs_chain=combine_docs_chain,
        question_generator=question_gen_chain,
    )
    
    destination_chains = {
        'retrieval': retrieval_chain,
    }
    
    main_chain = MultiPromptChain(
        router_chain=router_chain,
        destination_chains=destination_chains,
        default_chain=default_chain,
        verbose=True,
    )
```

 `make format`, `make lint` and `make test`
2023-09-28 15:59:25 -07:00
Nicolas
8ed013d278
docs: Mendable Search Improvements (#11199)
Improvements to the Mendable UI, more accurate responses, and bug fixes.
2023-09-28 15:57:04 -07:00
Piyush Jain
32d09bcd1e
Expanded version range for networkx, fixed sample notebook (#11094)
## Description
Expanded the upper bound for `networkx` dependency to allow installation
of latest stable version. Tested the included sample notebook with
version 3.1, and all steps ran successfully.
---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-09-28 15:33:30 -07:00
Piotr Mardziel
b40ecee4b9
FIx eval prompt (#11087)
**Description:** fixes a common typo in some of the eval criteria.
2023-09-28 15:21:15 -07:00
Guy Korland
5564833bd2
Add add_graph_documents support for FalkorDBGraph (#11122)
Adding `add_graph_documents` support for FalkorDBGraph and extending the
`Neo4JGraph` api so it can support `cypher.py`
2023-09-28 15:03:54 -07:00
Tomaz Bratanic
7d25a65b10
add from_existing_graph to neo4j vector (#11124)
This PR adds the option to create a Neo4jvector instance from existing
graph, which embeds existing text in the database and creates relevant
indices.
2023-09-28 15:02:26 -07:00
Noah Stapp
2c952de21a
Add support for MongoDB Atlas $vectorSearch vector search (#11139)
Adds support for the `$vectorSearch` operator for
MongoDBAtlasVectorSearch, which was announced at .Local London
(September 26th, 2023). This change maintains breaks compatibility
support for the existing `$search` operator used by the original
integration (https://github.com/langchain-ai/langchain/pull/5338) due to
incompatibilities in the Atlas search implementations.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-09-28 15:01:03 -07:00
Hugues
b599f91e33
LLMonitor Callback handler: fix bug (#11128)
Here is a small bug fix for the LLMonitor callback handler. I've also
added user identification capabilities.
2023-09-28 15:00:38 -07:00