Commit Graph

1611 Commits

Author SHA1 Message Date
了空
324e9c83d5
Add BiliBiliLoader to langchain.document_loaders.__init__.py (#2826) 2023-04-13 06:47:27 -07:00
Nuhman Pk
ed03e965de
Update README.md (#2805)
Added total download in a month (https://pepy.tech/project/langchain)
2023-04-12 22:02:06 -07:00
KullTC
64596b23b9
Return output of PythonAstREPLTool when falling back to exec() (#2780)
When the code ran by the PythonAstREPLTool contains multiple statements
it will fallback to exec() instead of using eval(). With this change, it
will also return the output of the code in the same way the
PythonREPLTool will.
2023-04-12 21:22:46 -07:00
Harrison Chase
1bb0706955
Harrison/comet ml (#2799)
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: Boris Feld <lothiraldan@gmail.com>
2023-04-12 21:21:51 -07:00
Harrison Chase
b2bc5ef56a
agent refactor (#2801) 2023-04-12 21:21:41 -07:00
Zach Jones
abfca72c0b
Add max_execution_time to openapi, pandas, and sql creators (#2779)
In #2399 we added the ability to set `max_execution_time` when creating
an AgentExecutor. This PR adds the `max_execution_time` argument to the
built-in pandas, sql, and openapi agents.

Co-authored-by: Zachary Jones <zjones@zetaglobal.com>
2023-04-12 17:09:42 -07:00
Matt Robinson
f0be3b0689
feat: add support for non-html in UnstructuredURLLoader (#2793)
### Summary

Adds support for processing non HTML document types in the URL loader.
For example, the URL loader can now process a PDF or markdown files
hosted at a URL.

### Testing

```python
from langchain.document_loaders import UnstructuredURLLoader

urls = ["https://www.understandingwar.org/sites/default/files/Russian%20Offensive%20Campaign%20Assessment%2C%20April%2011%2C%202023.pdf"]

loader = UnstructuredURLLoader(urls=urls, strategy="fast")
docs = loader.load()
print(docs[0].page_content[:1000])
```
2023-04-12 17:06:28 -07:00
Tim Connors
e081c62aac
Fixed k=0 bug on ConversationBufferWindowMemory (#2796)
Updated the "load_memory_variables" function of the
ConversationBufferWindowMemory to support a window size of 0 (k=0).
Previous behavior would return the full memory instead of an empty
array.
2023-04-12 17:05:54 -07:00
dev2049
a094b7f807
Improve eval chain prompt (#2798)
Eval chain is currently very sensitive to differences in phrasing,
punctuation, and tangential information. This prompt has worked better
for me on my examples.

More general q: Do we have any framework for evaluating default prompt
changes? Could maybe start doing some regression testing?
2023-04-12 17:05:20 -07:00
Kah Keng Tay
1c7fb31bba
Weaviate attributes and error handling (#2800) 2023-04-12 17:04:42 -07:00
dev2049
0e763677e4
Fix typo in qa eval chain prompt (#2797) 2023-04-12 14:17:25 -07:00
Harrison Chase
e49f1e628c
Harrison/gpt cache (#2744)
Co-authored-by: SimFG <bang.fu@zilliz.com>
2023-04-12 14:16:58 -07:00
Harrison Chase
425c437cd3 cr 2023-04-12 13:46:58 -07:00
Harrison Chase
a2d729e537 cr 2023-04-12 13:44:21 -07:00
Harrison Chase
7adbc4fbb4
agent memory (#2792) 2023-04-12 12:51:15 -07:00
Nuno Campos
1bea9ea4be
Fix async task being destroyed before cancelled (#2787) 2023-04-12 12:38:38 -07:00
Harrison Chase
819d72614a
version 138 (#2782) 2023-04-12 11:10:47 -07:00
wangml999
fa0c9390c2
Update custom_agent.ipynb (#2767)
Fixed an issue the agent is not taking the user's question as input.
2023-04-12 09:13:46 -07:00
Joshua Snyder
59d054308c
Add type inference for output parsers (#2769)
Currently, the output type of a number of OutputParser's `parse` methods
is `Any` when it can in fact be inferred.

This PR makes BaseOutputParser use a generic type and fixes the output
types of the following parsers:
- `PydanticOutputParser`
- `OutputFixingParser`
- `RetryOutputParser`
- `RetryWithErrorOutputParser`

The output of the `StructuredOutputParser` is corrected from `BaseModel`
to `Any` since there are no type guarantees provided by the parser.

Fixes issue #2715
2023-04-12 09:12:20 -07:00
Nuhman Pk
789cc314c5
Typo (#2747) 2023-04-12 09:06:30 -07:00
Harrison Chase
b92a89e29f cr 2023-04-11 23:52:14 -07:00
vowelparrot
94a92abf24
Add Retrieval Example for AI Plugins (#2737)
This PR proposes
- An NLAToolkit method to instantiate from an AI Plugin URL
- A notebook that shows how to use that alongside an example of using a
Retriever object to lookup specs and route queries to them on the fly

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-04-11 23:22:14 -07:00
Nuhman Pk
b5bbe601fb
Update chatgpt_plugins.ipynb (#2745)
Changed deprecated requests to requests_all in plugins example
2023-04-11 22:45:31 -07:00
Harrison Chase
b38a6ea7df
Harrison/apply llm flag (#2743)
Co-authored-by: Nick Gibb <gibbnick@gmail.com>
Co-authored-by: Nick Gibb <nick.gibb@bluedot.global>
2023-04-11 22:02:37 -07:00
vr140
dd59193757
Remove unnecessary method from Qdrant vectorstore and clean up docstrings (#2700)
**Problem:**

The `from_documents` method in Qdrant vectorstore is unnecessary because
it does not change any default behavior from the abstract base class
method of `from_documents` (contrast this with the method in Chroma
which makes a change from default and turns `embeddings` into an
Optional parameter).

Also, the docstrings need some cleanup.

**Solution:**

Remove unnecessary method and improve docstrings.

---------

Co-authored-by: Vijay Rajaram <vrajaram3@gatech.edu>
2023-04-11 21:34:22 -07:00
Matthew Plachter
933dfac583
Add Zapier NLA OAuth access_token to be used (#2726)
This change allows the user to initialize the ZapierNLAWrapper with a
valid Zapier NLA OAuth Access_Token, which would be used to make
requests back to the Zapier NLA API.

When a `zapier_nla_oauth_access_token` is passed to the ZapierNLAWrapper
it is no longer required for the `ZAPIER_NLA_API_KEY ` environment
variable to be set, still having it set will not affect the behavior as
the `zapier_nla_oauth_access_token` will be used over the
`ZAPIER_NLA_API_KEY`
2023-04-11 21:32:54 -07:00
Harrison Chase
507cee5ee5
Harrison/pinecone hybrid update (#2742)
Co-authored-by: acatav <39461369+acatav@users.noreply.github.com>
Co-authored-by: Amnon Catav <catav.amnon1@gmail.com>
2023-04-11 21:32:17 -07:00
Johnny Lee
744c25cd0a
Updating YoutubeLoader.from_youtube_channel name and doc to reflect actual usage (#2734)
the function actually updates video_id from URL not channel.

The docs still reflect the previous old function name
`from_youtube_url`. Resolves #1962


https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/youtube.html
2023-04-11 21:12:58 -07:00
Johnny Lee
0ab364404e
add continue to fix 'continue_on_failure' parameter for URL doc loader (#2735)
Currently, the function still fails if `continue_on_failure` is set to
True, because `elements` is not set.

---------

Co-authored-by: leecjohnny <johnny-lee1255@users.noreply.github.com>
2023-04-11 21:12:39 -07:00
sergerdn
4bdcedab54
fix: some imports for integration tests (#2612)
Add more missed imports for integration tests. Bump `pytest` to the
current latest version.
Fix `tests/integration_tests/vectorstores/test_elasticsearch.py` to
update its cassette(easy fix).

Related PR: https://github.com/hwchase17/langchain/pull/2560
2023-04-11 20:45:36 -07:00
Ankush Gola
c1521ddbdb
Add workaround for not having async vector store methods (#2733)
This allows us to use the async API for the Retrieval chains, though it is not guaranteed to be thread safe.
2023-04-11 18:49:08 -07:00
vowelparrot
0806951c07
Update VectorStore Class Method Typing (#2731)
Avoid using placeholder methods that only perform a `cast()`
operation because the typing would otherwise be inferred to be the
parent `VectorStore` class. This is unnecessary with TypeVar's.
2023-04-11 14:14:49 -07:00
Adam McCabe
446c3d586c
Add PATCH and DELETE to OpenAPI Agent (#2729)
This PR proposes an update to the OpenAPI Planner and Planner Prompts to
make Patch and Delete available to the planner and executor. I followed
the same patterns as for GET and POST, and made some updates to the
examples available to the Planner and Orchestrator.

Of note, I tried to write prompts for DELETE such that the model will
only execute that job if the User specifically asks for a 'Delete' (see
the Prompt_planner.py examples to see specificity), or if the User had
previously authorized the Delete in the Conversation memory. Although
PATCH also modifies existing data, I considered it lower risk and so did
not try to enforce the same restrictions on the Planner.
2023-04-11 13:26:04 -07:00
vinoyang
8073bc849f
Minor: Remove duplicated word in error message (#2706)
Removed the duplicated word "it" from the error message.
From:
`Please it install it with xxx`
To:
`Please install it with xxx`.
2023-04-11 13:10:33 -07:00
134ARG
1e60e6e15b
Fix the unset argument in calling llama model (#2714)
When using the llama.cpp together with agent like
zero-shot-react-description, the missing branch will cause the parameter
`stop` left empty, resulting in unexpected output format from the model.

This patch fixes that issue.
2023-04-11 11:02:39 -07:00
Joshua Snyder
f435f2267c
Use tiktoken for Python 3.8 (#2709)
Fixes issue #2677

`tiktoken` is supported for Python 3.8, so there is no need to use the
fallback GPT-2 tokenizer.
2023-04-11 11:02:28 -07:00
Kei Kamikawa
186ca9d3e4
fixed aiohttp.client_exceptions.ClientConnectionError: Connection closed (#2718)
I fixed an issue where an error would always occur when making a request
using the `TextRequestsWrapper` with async API.

This is caused by escaping the scope of the context, which causes the
connection to be broken when reading the response body.

The correct usage is as described in the [official
tutorial](https://docs.aiohttp.org/en/stable/client_quickstart.html#make-a-request),
where the text method must also be handled in the context scope.

<details>

<summary>Stacktrace</summary>

```
  File "/home/vscode/.cache/pypoetry/virtualenvs/codehex-workspace-xS3fZVNL-py3.11/lib/python3.11/site-packages/langchain/tools/base.py", line 116, in arun
    raise e
  File "/home/vscode/.cache/pypoetry/virtualenvs/codehex-workspace-xS3fZVNL-py3.11/lib/python3.11/site-packages/langchain/tools/base.py", line 110, in arun
    observation = await self._arun(tool_input)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.cache/pypoetry/virtualenvs/codehex-workspace-xS3fZVNL-py3.11/lib/python3.11/site-packages/langchain/agents/tools.py", line 22, in _arun
    return await self.coroutine(tool_input)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.cache/pypoetry/virtualenvs/codehex-workspace-xS3fZVNL-py3.11/lib/python3.11/site-packages/langchain/chains/base.py", line 234, in arun
    return (await self.acall(args[0]))[self.output_keys[0]]
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.cache/pypoetry/virtualenvs/codehex-workspace-xS3fZVNL-py3.11/lib/python3.11/site-packages/langchain/chains/base.py", line 154, in acall
    raise e
  File "/home/vscode/.cache/pypoetry/virtualenvs/codehex-workspace-xS3fZVNL-py3.11/lib/python3.11/site-packages/langchain/chains/base.py", line 148, in acall
    outputs = await self._acall(inputs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/src/tools/example.py", line 153, in _acall
    api_response = await self.requests_wrapper.aget("http://example.com")
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.cache/pypoetry/virtualenvs/codehex-workspace-xS3fZVNL-py3.11/lib/python3.11/site-packages/langchain/requests.py", line 130, in aget
    return await response.text()
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.cache/pypoetry/virtualenvs/codehex-workspace-xS3fZVNL-py3.11/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1081, in text
    await self.read()
  File "/home/vscode/.cache/pypoetry/virtualenvs/codehex-workspace-xS3fZVNL-py3.11/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1037, in read
    self._body = await self.content.read()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.cache/pypoetry/virtualenvs/codehex-workspace-xS3fZVNL-py3.11/lib/python3.11/site-packages/aiohttp/streams.py", line 349, in read
  raise self._exception
aiohttp.client_exceptions.ClientConnectionError: Connection closed
```

</details>
2023-04-11 10:52:55 -07:00
Dogan Can Bakir
3623bdb31b
Make the OpenAPI agent's verbose print optional (#2666) 2023-04-11 10:42:39 -07:00
vowelparrot
709f26b69e
Added bilibili loader (#2673) (#2724)
I've added a bilibili loader, bilibili is a very active video site in
China and I think we need this loader.

Example:
```python
from langchain.document_loaders.bilibili import BiliBiliLoader

loader = BiliBiliLoader(
       ["https://www.bilibili.com/video/BV1xt411o7Xu/",
       "https://www.bilibili.com/video/av330407025/"]
)
docs = loader.load()
```

Co-authored-by: 了空 <568250549@qq.com>
2023-04-11 10:40:32 -07:00
David Wu
d42deff402
fixed typo (#2720)
changed "to" to "too" in the memory notebook
2023-04-11 09:53:38 -07:00
David Wu
263ce40844
added a missing word (typo) (#2719)
Changed from "You may often to" to "You may often have to" to fix the
sentence.
2023-04-11 09:09:28 -07:00
Harrison Chase
66786b0f0f cr 2023-04-11 08:16:06 -07:00
Harrison Chase
948b14b52a
agents docs and version bump (#2717) 2023-04-11 08:08:43 -07:00
Abhik Singla
955bd2e1db
Fixed Ast Python Repl for Chatgpt multiline commands (#2406)
Resolves issue https://github.com/hwchase17/langchain/issues/2252

---------

Co-authored-by: Abhik Singla <abhiksingla@microsoft.com>
2023-04-10 21:25:03 -07:00
Harrison Chase
1271c00ff0
Harrison/openapi planner (#2692)
Co-authored-by: Adam McCabe <adam.r.mccabe@gmail.com>
2023-04-10 21:22:42 -07:00
Harrison Chase
e0a13e9355
Harrison/postgres (#2691)
Co-authored-by: Ankit Jain <ankneo@users.noreply.github.com>
2023-04-10 21:15:42 -07:00
Guohao Li
bb5118f4c9
Add notebook example for camel role playing (#2689)
This PR adds a LangChain implementation of CAMEL role-playing example:
https://github.com/lightaime/camel.

I am sorry that I am not that familiar with LangChain. So I only
implement it in a naive way. There may be a better way to implement it.
2023-04-10 21:12:45 -07:00
Harrison Chase
d3f779d61d
baby agi agent (#2648)
Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
2023-04-10 21:03:30 -07:00
Naveen Tatikonda
4364d3316e
Add custom vector fields and text fields for OpenSearch (#2652)
**Description**
Add custom vector field name and text field name while indexing and
querying for OpenSearch

**Issues**
https://github.com/hwchase17/langchain/issues/2500

Signed-off-by: Naveen Tatikonda <navtat@amazon.com>
2023-04-10 21:02:02 -07:00
Pavel Shibanov
023de9a70b
Add OpenAIEmbeddings special token params for tiktoken (#2682)
#2681 

Original type hints
```python
allowed_special: Union[Literal["all"], AbstractSet[str]] = set(),  # noqa: B006
disallowed_special: Union[Literal["all"], Collection[str]] = "all",
```
from

46287bfa49/tiktoken/core.py (L79-L80)
are not compatible with pydantic

<img width="718" alt="image"
src="https://user-images.githubusercontent.com/5096640/230993236-c744940e-85fb-4baa-b9da-8b00fb60a2a8.png">

I think we could use
```python
allowed_special: Union[Literal["all"], Set[str]] = set()
disallowed_special: Union[Literal["all"], Set[str], Tuple[()]] = "all"
```

Please let me know if you would like to implement it differently.
2023-04-10 21:00:55 -07:00