Commit Graph

1584 Commits

Author SHA1 Message Date
jerwelborn
55efbb8a7e
pydantic/json parsing (#1722)
```
class Joke(BaseModel):
    setup: str = Field(description="question to set up a joke")
    punchline: str = Field(description="answer to resolve the joke")

joke_query = "Tell me a joke."

# Or, an example with compound type fields.
#class FloatArray(BaseModel):
#    values: List[float] = Field(description="list of floats")
#
#float_array_query = "Write out a few terms of fiboacci."

model = OpenAI(model_name='text-davinci-003', temperature=0.0)
parser = PydanticOutputParser(pydantic_object=Joke)
prompt = PromptTemplate(
    template="Answer the user query.\n{format_instructions}\n{query}\n",
    input_variables=["query"],
    partial_variables={"format_instructions": parser.get_format_instructions()}
)

_input = prompt.format_prompt(query=joke_query)
print("Prompt:\n", _input.to_string())
output = model(_input.to_string())
print("Completion:\n", output)
parsed_output = parser.parse(output)
print("Parsed completion:\n", parsed_output)
```

```
Prompt:
 Answer the user query.
The output should be formatted as a JSON instance that conforms to the JSON schema below.  For example, the object {"foo":  ["bar", "baz"]} conforms to the schema {"foo": {"description": "a list of strings field", "type": "string"}}.

Here is the output schema:
---
{"setup": {"description": "question to set up a joke", "type": "string"}, "punchline": {"description": "answer to resolve the joke", "type": "string"}}
---

Tell me a joke.

Completion:
 {"setup": "Why don't scientists trust atoms?", "punchline": "Because they make up everything!"}

Parsed completion:
 setup="Why don't scientists trust atoms?" punchline='Because they make up everything!'
```

Ofc, works only with LMs of sufficient capacity. DaVinci is reliable but
not always.

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-03-16 21:43:11 -07:00
Jonathan Pedoeem
606605925d
Adding ability to return_pl_id to all PromptLayer Models in LangChain (#1699)
PromptLayer now has support for [several different tracking
features.](https://magniv.notion.site/Track-4deee1b1f7a34c1680d085f82567dab9)
In order to use any of these features you need to have a request id
associated with the request.

In this PR we add a boolean argument called `return_pl_id` which will
add `pl_request_id` to the `generation_info` dictionary associated with
a generation.

We also updated the relevant documentation.
2023-03-16 17:05:23 -07:00
Harrison Chase
3ea6d9c4d2
add docs for save/load messages (#1697) 2023-03-15 13:13:08 -07:00
Piyush Jain
1279c8de39
Fixed typo, clarified language (#1682) 2023-03-15 08:00:11 -07:00
at-b612
c7779c800a
Added Mynd URL to gallery (#1684) 2023-03-15 07:59:59 -07:00
Jithin James
6f4f771897
docs: add path to state_of_the_union.txt in indexes/getting_started page (#1691)
add the state_of_the_union.txt file so that its easier to follow through
with the example.

---------

Co-authored-by: Jithin James <jjmachan@pop-os.localdomain>
2023-03-15 07:59:47 -07:00
Ankush Gola
d4edd3c312
Zapier Integration (#1654)
* Zapier Wrapper and Tools (implemented by Zapier Team)
* Zapier Toolkit, examples with mrkl agent

---------

Co-authored-by: Mike Knoop <mikeknoop@gmail.com>
Co-authored-by: Robert Lewis <robert.lewis@zapier.com>
2023-03-14 23:06:17 -07:00
Harrison Chase
0b29e68c17
Harrison/pgvector (#1679)
Co-authored-by: Aman Kumar <krsingh.aman@gmail.com>
2023-03-14 21:13:58 -07:00
Harrison Chase
4d7fdb8957
Harrison/gml save (#1676)
Co-authored-by: Satoru Sakamoto <51464932+satoru814@users.noreply.github.com>
2023-03-14 20:00:22 -07:00
Harrison Chase
656efe6ef3
Harrison/fix nb (#1678) 2023-03-14 19:34:23 -07:00
Matt Robinson
63aa28e2a6
feat: allow the unstructured kwargs to be passed in to Unstructured document loaders (#1667)
### Summary

Allows users to pass in `**unstructured_kwargs` to Unstructured document
loaders. Implemented with the `strategy` kwargs in mind, but will pass
in other kwargs like `include_page_breaks` as well. The two currently
supported strategies are `"hi_res"`, which is more accurate but takes
longer, and `"fast"`, which processes faster but with lower accuracy.
The `"hi_res"` strategy is the default. For PDFs, if `detectron2` is not
available and the user selects `"hi_res"`, the loader will fallback to
using the `"fast"` strategy.


### Testing

#### Make sure the `strategy` kwarg works

Run the following in iPython to verify that the `"fast"` strategy is
indeed faster.

```python
from langchain.document_loaders import UnstructuredFileLoader

loader = UnstructuredFileLoader("layout-parser-paper-fast.pdf", strategy="fast", mode="elements")
%timeit loader.load()

loader = UnstructuredFileLoader("layout-parser-paper-fast.pdf", mode="elements")
%timeit loader.load()
```

On my system I get:

```python
In [3]: from langchain.document_loaders import UnstructuredFileLoader

In [4]: loader = UnstructuredFileLoader("layout-parser-paper-fast.pdf", strategy="fast", mode="elements")

In [5]: %timeit loader.load()
247 ms ± 369 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [6]: loader = UnstructuredFileLoader("layout-parser-paper-fast.pdf", mode="elements")

In [7]: %timeit loader.load()
2.45 s ± 31 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

#### Make sure older versions of `unstructured` still work

Run `pip install unstructured==0.5.3` and then verify the following runs
without error:

```python
from langchain.document_loaders import UnstructuredFileLoader

loader = UnstructuredFileLoader("layout-parser-paper-fast.pdf",  mode="elements")
loader.load()
```
2023-03-14 18:15:28 -07:00
Matthias Kern
c3dfbdf0da
Remove outdated code from Chat VectorDB QA example (#1670) 2023-03-14 18:13:51 -07:00
Bilel MEDIMEGH
a2280f321f
Docs: Fix typo in memory/key_concepts.md (#1671)
dialouge -> dialogue
2023-03-14 18:12:01 -07:00
Xin Qiu
4e13cef05a
feat: add redisearch vectorstore (#1307)
# Description

Add `RediSearch` vectorstore for LangChain

RediSearch: [RediSearch quick
start](https://redis.io/docs/stack/search/quick_start/)

# How to use

```
from langchain.vectorstores.redisearch import RediSearch

rds = RediSearch.from_documents(docs, embeddings,redisearch_url="redis://localhost:6379")
```
2023-03-14 18:06:03 -07:00
Harrison Chase
2d098e8869
Harrison/agent eval (#1620)
Co-authored-by: jerwelborn <jeremy.welborn@gmail.com>
2023-03-14 12:37:48 -07:00
Harrison Chase
7cf46b3fee
Harrison/convo agent (#1642) 2023-03-14 09:42:24 -07:00
Jon Luo
0a1b1806e9
sql: do not hard code the LIMIT clause in the table_info section (#1563)
Seeing a lot of issues in Discord in which the LLM is not using the
correct LIMIT clause for different SQL dialects. ie, it's using `LIMIT`
for mssql instead of `TOP`, or instead of `ROWNUM` for Oracle, etc.
I think this could be due to us specifying the LIMIT statement in the
example rows portion of `table_info`. So the LLM is seeing the `LIMIT`
statement used in the prompt.
Since we can't specify each dialect's method here, I think it's fine to
just replace the `SELECT... LIMIT 3;` statement with `3 rows from
table_name table:`, and wrap everything in a block comment directly
following the `CREATE` statement. The Rajkumar et al paper wrapped the
example rows and `SELECT` statement in a block comment as well anyway.
Thoughts @fpingham?
2023-03-13 23:08:27 -07:00
Tim Asp
b3234bf3b0
cleanup: unify 3 different pdf loaders, rename PagedPDFSplitter (#1615)
`OnlinePDFLoader` and `PagedPDFSplitter` lived separate from the rest of
the pdf loaders.

Because they're all similar, I propose moving all to `pdy.py` and the
same docs/examples page.

Additionally, `PagedPDFSplitter` naming doesn't match the pattern the
rest of the loaders follow, so I renamed to `PyPDFLoader` and had it
inherit from `BasePDFLoader` so it can now load from remote file
sources.
2023-03-13 23:06:50 -07:00
Harrison Chase
56aff797c0
docs req (#1647) 2023-03-13 16:03:32 -07:00
Harrison Chase
d53ff270e0
bump version to 109 (#1646) 2023-03-13 15:52:35 -07:00
Harrison Chase
df6c33d4b3
Harrison/new output parser (#1617) 2023-03-13 15:08:39 -07:00
Eugene Yurtsev
bd4a2a670b
Add copy button to sphinx notebooks (#1622)
This adds a copy button at the top right corner of all notebook cells in
sphinx
notebooks.
2023-03-12 21:15:07 -07:00
Ikko Eltociear Ashimine
6e98ab01e1
Fix typo in vectorstore.ipynb (#1614)
Initalize -> Initialize
2023-03-12 14:12:47 -07:00
yakigac
acd86d33bc
Add read only shared memory (#1491)
Provide shared memory capability for the Agent.
Inspired by #1293 .

## Problem

If both Agent and Tools (i.e., LLMChain) use the same memory, both of
them will save the context. It can be annoying in some cases.


## Solution

Create a memory wrapper that ignores the save and clear, thereby
preventing updates from Agent or Tools.
2023-03-12 09:34:36 -07:00
Harrison Chase
c9b5a30b37
move output parsing (#1605) 2023-03-11 16:41:03 -08:00
Harrison Chase
15de3e8137
Harrison/docs footer (#1600)
Co-authored-by: Albert Avetisian <albert.avetisian@gmail.com>
2023-03-11 09:18:35 -08:00
Harrison Chase
9f78717b3c
Harrison/callbacks (#1587) 2023-03-10 12:53:09 -08:00
Harrison Chase
90846dcc28
fix chat agent (#1586) 2023-03-10 12:40:37 -08:00
Zach Schillaci
624c72c266
Add wikipedia tool doc (#1579) 2023-03-10 07:07:27 -08:00
Tim Asp
30383abb12
Add CSVLoader document loader (#1573)
Simple CSV document loader which wraps `csv` reader, and preps the file
with a single `Document` per row.

The column header is prepended to each value for context which is useful
for context with embedding and semantic search
2023-03-09 16:35:18 -08:00
Andriy Mulyar
c9189d354a
AtlasDB vector store documentation updates. (#1572)
- Updated errors in the AtlasDB vector store documentation
- Removed extraneous output logs in example notebook.
2023-03-09 16:31:14 -08:00
Matt Robinson
7018806a92
feat: document loader for markdown files (#1558)
### Summary

Adds a document loader for handling markdown files. This document loader
requires `unstructured>=0.4.16`.

### Testing

```python
from langchain.document_loaders import UnstructuredMarkdownLoader

loader = UnstructuredMarkdownLoader("README.md")
loader.load()
```
2023-03-09 10:55:07 -08:00
Harrison Chase
bd335ffd64
bump version to 106 (#1562) 2023-03-09 10:20:54 -08:00
Harrison Chase
a094c49153
add chat agent (#1509) 2023-03-09 09:12:08 -08:00
Brenton Wheeler
99fe023496
docs: fix typo in modules/indexes/chain_examples/question_answering (#1551)
docs: fix typo in modules/indexes/chain_examples/question_answering


![image](https://user-images.githubusercontent.com/11394076/224007874-3a52adf6-ff7a-4f22-9dbf-18c83d08167f.png)
2023-03-09 09:11:43 -08:00
Harrison Chase
3ee32a01ea
Harrison/prompt layer (#1547)
Co-authored-by: Jonathan Pedoeem <jonathanped@gmail.com>
Co-authored-by: AbuBakar <abubakarsohail123@gmail.com>
2023-03-08 21:24:27 -08:00
Harrison Chase
cc423f40f1
Harrison/youtube loader (#1545)
Co-authored-by: Julian Wustl <57504258+Julianwustl@users.noreply.github.com>
2023-03-08 20:53:27 -08:00
Harrison Chase
523ad8d2e2
Harrison/chat history formatter1 (#1538)
Co-authored-by: Youssef A. Abukwaik <yousseb@users.noreply.github.com>
2023-03-08 20:46:37 -08:00
Graham Neubig
31303d0b11
Added other evaluation metrics for data-augmented QA (#1521)
This PR adds additional evaluation metrics for data-augmented QA,
resulting in a report like this at the end of the notebook:

![Screen Shot 2023-03-08 at 8 53 23
AM](https://user-images.githubusercontent.com/398875/223731199-8eb8e77f-5ff3-40a2-a23e-f3bede623344.png)

The score calculation is based on the
[Critique](https://docs.inspiredco.ai/critique/) toolkit, an API-based
toolkit (like OpenAI) that has minimal dependencies, so it should be
easy for people to run if they choose.

The code could further be simplified by actually adding a chain that
calls Critique directly, but that probably should be saved for another
PR if necessary. Any comments or change requests are welcome!
2023-03-08 20:41:03 -08:00
gidler
494c9d341a
[DOCS] Assorted wording, punctuation, and consistency revisions (#1443)
Contributing some small fixes I noticed while reading through the
documentation.

Thank you for a creating and maintaining this project!
2023-03-08 20:16:09 -08:00
Harrison Chase
c4a557bdd4
add concept of prompt collection (#1507) 2023-03-08 08:31:29 -08:00
Ivan
97e3666e0d
changed requests.run to requests.get (#1485)
This pull request proposes an update to the Lightweight wrapper
library's documentation. The current documentation provides an example
of how to use the library's requests.run method, as follows:
requests.run("https://www.google.com"). However, this example does not
work for the 0.0.102 version of the library.

Testing:

The changes have been tested locally to ensure they are working as
intended.

Thank you for considering this pull request.
2023-03-07 21:10:23 -08:00
Tom Dyson
e3354404ad
Fix link to Pinecone notebook (#1492) 2023-03-07 15:24:03 -08:00
Harrison Chase
3610ef2830
add fake embeddings class (#1503) 2023-03-07 15:23:46 -08:00
Harrison Chase
4f41e20f09
memory docs (#1501) 2023-03-07 11:02:46 -08:00
Harrison Chase
f276bfad8e
Harrison/chat memory (#1495) 2023-03-07 09:02:40 -08:00
Harrison Chase
7bec461782
Harrison/memory refactor (#1478)
moves memory to own module, factors out common stuff
2023-03-07 07:59:37 -08:00
Harrison Chase
0e21463f07
(rfc) chat models (#1424)
Co-authored-by: Ankush Gola <ankush.gola@gmail.com>
2023-03-06 08:34:24 -08:00
Harrison Chase
63a5614d23
Harrison/simple memory (#1435)
Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>
2023-03-04 08:15:52 -08:00
Harrison Chase
a1b9dfc099
Harrison/similarity search chroma (#1434)
Co-authored-by: shibuiwilliam <shibuiyusuke@gmail.com>
2023-03-04 08:10:15 -08:00
Ikko Eltociear Ashimine
b8a7828d1f
Update huggingface_datasets.ipynb (#1417)
HuggingFace -> Hugging Face
2023-03-04 00:22:31 -08:00
Tim Asp
23231d65a9
Add PyMuPDF PDF loader (#1426)
Different PDF libraries have different strengths and weaknesses. PyMuPDF
does a good job at extracting the most amount of content from the doc,
regardless of the source quality, extremely fast (especially compared to
Unstructured).

https://pymupdf.readthedocs.io/en/latest/index.html
2023-03-03 20:59:28 -08:00
3d54b05863
searx: add install instructions, update doc and notebooks (#1420)
- Added instructions on setting up self hosted searx
- Add notebook example with agent
- Use `localhost:8888` as example url to stay consistent since public
instances are not really usable.

Co-authored-by: blob42 <spike@w530>
2023-03-03 20:57:50 -08:00
Tim Asp
bca0935d90
[docs] fix minor import error (#1425) 2023-03-03 16:10:07 -08:00
JonLuca De Caro
443992c4d5
[Docs] Add missing word from prompt docs (#1406)
The prompt in the first example of the quickstart guide was missing `for
`
2023-03-02 16:02:54 -08:00
Jason Gill
1989e7d4c2
Update examples to prevent confusing missing _type warning (#1391)
The YAML and JSON examples of prompt serialization now give a strange
`No '_type' key found, defaulting to 'prompt'` message when you try to
run them yourself or copy the format of the files. The reason for this
harmless warning is that the _type key was not in the config files,
which means they are parsed as a standard prompt.

This could be confusing to new users (like it was confusing to me after
upgrading from 0.0.85 to 0.0.86+ for my few_shot prompts that needed a
_type added to the example_prompt config), so this update includes the
_type key just for clarity.

Obviously this is not critical as the warning is harmless, but it could
be confusing to track down or be interpreted as an error by a new user,
so this update should resolve that.
2023-03-02 07:39:57 -08:00
Harrison Chase
dda5259f68
bump version to 0.0.99 (#1390) 2023-03-02 07:25:59 -08:00
Kacper Łukawski
9ac442624c
Add Qdrant named arguments (#1386)
This PR:
- Increases `qdrant-client` version to 1.0.4
- Introduces custom content and metadata keys (as requested in #1087)
- Moves all the `QdrantClient` parameters into the method parameters to
simplify code completion
2023-03-02 07:05:14 -08:00
Ankush Gola
fe30be6fba
add async and streaming support to OpenAIChat (#1378)
title says it all
2023-03-01 21:55:43 -08:00
Lakshya Agarwal
cfed0497ac
Minor grammatical fixes (#1325)
Fixed typos and links in a few places across documents
2023-03-01 21:18:09 -08:00
Harrison Chase
1cd8996074
Harrison/summarizer chain (#1356)
Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>
2023-03-01 20:59:07 -08:00
Harrison Chase
4b5e850361
chatgpt wrapper (#1367) 2023-03-01 11:47:01 -08:00
Harrison Chase
4d4b43cf5a
fix doc names (#1354) 2023-03-01 09:40:31 -08:00
Harrison Chase
fe7dbecfe6
pandas and csv agents (#1353) 2023-02-28 22:19:11 -08:00
Harrison Chase
02ec72df87
improve docs (#1351) 2023-02-28 21:37:18 -08:00
Jon Luo
92ab27e4b8
sql doc formatting (#1350)
My bad, missed a few tabs between the two PRs
2023-02-28 19:54:46 -08:00
Ankush Gola
82baecc892
Add a SQL agent for interacting with SQL Databases and JSON Agent for interacting with large JSON blobs (#1150)
This PR adds 

* `ZeroShotAgent.as_sql_agent`, which returns an agent for interacting
with a sql database. This builds off of `SQLDatabaseChain`. The main
advantages are 1) answering general questions about the db, 2) access to
a tool for double checking queries, and 3) recovering from errors
* `ZeroShotAgent.as_json_agent` which returns an agent for interacting
with json blobs.
* Several examples in notebooks

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-02-28 19:44:39 -08:00
Jon Luo
35f1e8f569
separate columns by tabs instead of single space in sql sample rows (#1348)
Use tabs to separate columns instead of a single space - confusing when
there are spaces in a cell
2023-02-28 18:59:53 -08:00
James Brotchie
3574418a40
Fix link in summarization.md (#1344)
"Utilities for working with Documents" was linking to a non-useful page.
Re-linked to the utils page that includes info about working with docs.
2023-02-28 18:58:12 -08:00
Jon Luo
5bf8772f26
add option to use user-defined SQL table info (#1347)
Currently, table information is gathered through SQLAlchemy as complete
table DDL and a user-selected number of sample rows from each table.
This PR adds the option to use user-defined table information instead of
automatically collecting it. This will use the provided table
information and fall back to the automatic gathering for tables that the
user didn't provide information for.

Off the top of my head, there are a few cases where this can be quite
useful:
- The first n rows of a table are uninformative, or very similar to one
another. In this case, hand-crafting example rows for a table such that
they provide the good, diverse information can be very helpful. Another
approach we can think about later is getting a random sample of n rows
instead of the first n rows, but there are some performance
considerations that need to be taken there. Even so, hand-crafting the
sample rows is useful and can guarantee the model sees informative data.
- The user doesn't want every column to be available to the model. This
is not an elegant way to fulfill this specific need since the user would
have to provide the table definition instead of a simple list of columns
to include or ignore, but it does work for this purpose.
- For the developers, this makes it a lot easier to compare/benchmark
the performance of different prompting structures for providing table
information in the prompt.

These are cases I've run into myself (particularly cases 1 and 3) and
I've found these changes useful. Personally, I keep custom table info
for a few tables in a yaml file for versioning and easy loading.

Definitely open to other opinions/approaches though!
2023-02-28 18:58:04 -08:00
Harrison Chase
786852e9e6
partial variables (#1308) 2023-02-28 08:40:35 -08:00
Tim Asp
72ef69d1ba
Add new iFixit document loader (#1333)
iFixit is a wikipedia-like site that has a huge amount of open content
on how to fix things, questions/answers for common troubleshooting and
"things" related content that is more technical in nature. All content
is licensed under CC-BY-SA-NC 3.0

Adding docs from iFixit as context for user questions like "I dropped my
phone in water, what do I do?" or "My macbook pro is making a whining
noise, what's wrong with it?" can yield significantly better responses
than context free response from LLMs.
2023-02-27 20:40:20 -08:00
Matt Robinson
1aa41b5741
feat: document loader for image files (#1330)
### Summary

Adds a document loader for image files such as `.jpg` and `.png` files.

### Testing

Run the following using the example document from the [`unstructured`
repo](https://github.com/Unstructured-IO/unstructured/tree/main/example-docs).

```python
from langchain.document_loaders.image import UnstructuredImageLoader

loader = UnstructuredImageLoader("layout-parser-paper-fast.jpg")
loader.load()
```
2023-02-27 14:43:32 -08:00
Eugene Yurtsev
c14cff60d0
Documentation: Minor typo fixes (#1327)
Fixing a few minor typos in the documentation (and likely introducing
other
ones in the process).
2023-02-27 14:40:43 -08:00
Harrison Chase
f61858163d
bump version to 0.0.95 (#1324) 2023-02-27 07:45:54 -08:00
Harrison Chase
0824d65a5c
Harrison/indexing pipeline (#1317) 2023-02-27 00:31:36 -08:00
Akshay
a0bf856c70
Update agent_vectorstore.ipynb (#1318)
nitpicking but just thought i'd add this typo which I found when going
through the How-to 😄 (unless it was intentional) also, it's amazing that
you added ReAct to LangChain!
2023-02-26 23:22:35 -08:00
Harrison Chase
166cda2cc6
Harrison/deeplake (#1316)
Co-authored-by: Davit Buniatyan <d@activeloop.ai>
2023-02-26 22:35:04 -08:00
Harrison Chase
aaad6cc954
Harrison/atlas db (#1315)
Co-authored-by: Brandon Duderstadt <brandonduderstadt@gmail.com>
2023-02-26 22:11:38 -08:00
Marc Puig
3989c793fd
Making it possible to use "certainty" as a parameter for the weaviate similarity_search (#1218)
Checking if weaviate similarity_search kwargs contains "certainty" and
use it accordingly. The minimal level of certainty must be a float, and
it is computed by normalized distance.
2023-02-26 17:55:28 -08:00
Harrison Chase
81abcae91a
Harrison/banana fix (#1311)
Co-authored-by: Erik Dunteman <44653944+erik-dunteman@users.noreply.github.com>
2023-02-26 17:53:57 -08:00
Casey A. Fitzpatrick
648b3b3909
Fix use case sentence for bash util doc (#1295)
Thanks for all your hard work!

I noticed a small typo in the bash util doc so here's a quick update.
Additionally, my formatter caught some spacing in the `.md` as well.
Happy to revert that if it's an issue.

The main change is just
```
- A common use case this is for letting it interact with your local file system. 

+ A common use case for this is letting the LLM interact with your local file system.
```

## Testing

`make docs_build` succeeds locally and the changes show as expected ✌️ 
<img width="704" alt="image"
src="https://user-images.githubusercontent.com/17773666/221376160-e99e59a6-b318-49d1-a1d7-89f5c17cdab4.png">
2023-02-26 17:41:03 -08:00
Ingo Kleiber
fd9975dad7
add CoNLL-U document loader (#1297)
I've added a simple
[CoNLL-U](https://universaldependencies.org/format.html) document
loader. CoNLL-U is a common format for NLP tasks and is used, for
example, in the Universal Dependencies treebank corpora. The loader
reads a single file in standard CoNLL-U format and returns a document.
2023-02-26 17:27:00 -08:00
Harrison Chase
d29f74114e
copy paste loader (#1302) 2023-02-26 17:26:37 -08:00
Harrison Chase
ce441edd9c
improve docs (#1309) 2023-02-26 11:25:16 -08:00
Harrison Chase
6f30d68581
add example of using agent with vectorstores (#1285) 2023-02-25 13:27:24 -08:00
Matt Robinson
2f15c11b87
feat: document loader for MS Word documents (#1282)
### Summary

Adds a document loader for MS Word Documents. Works with both `.docx`
and `.doc` files as longer as the user has installed
`unstructured>=0.4.11`.

### Testing

The follow workflow test the loader for both `.doc` and `.docx` files
using example docs from the `unstructured` repo.

#### `.docx`

```python
from langchain.document_loaders import UnstructuredWordDocumentLoader

filename = "../unstructured/example-docs/fake.docx"
loader = UnstructuredWordDocumentLoader(filename)
loader.load()
```

#### `.doc`

```python
from langchain.document_loaders import UnstructuredWordDocumentLoader

filename = "../unstructured/example-docs/fake.doc"
loader = UnstructuredWordDocumentLoader(filename)
loader.load()
```
2023-02-24 08:26:19 -08:00
Harrison Chase
96db6ed073
cleanup (#1274) 2023-02-24 07:38:24 -08:00
Harrison Chase
42167a1e24
Harrison/fb loader (#1277)
Co-authored-by: Vairo Di Pasquale <vairo.dp@gmail.com>
2023-02-24 07:22:48 -08:00
Klein Tahiraj
8a0751dadd
adding .ipynb loader and documentation Fixes #1248 (#1252)
`NotebookLoader.load()` loads the `.ipynb` notebook file into a
`Document` object.

**Parameters**:

* `include_outputs` (bool): whether to include cell outputs in the
resulting document (default is False).
* `max_output_length` (int): the maximum number of characters to include
from each cell output (default is 10).
* `remove_newline` (bool): whether to remove newline characters from the
cell sources and outputs (default is False).
* `traceback` (bool): whether to include full traceback (default is
False).
2023-02-24 07:10:35 -08:00
Enrico Shippole
9becdeaadf
Add Writer, Banana, Modal, StochasticAI (#1270)
Add LLM wrappers and examples for Banana, Writer, Modal, Stochastic AI

Added rigid json format for Banana and Modal
2023-02-24 06:58:58 -08:00
Matt Robinson
10e73a3723
docs: remove nltk download steps (#1253)
### Summary

Updates the docs to remove the `nltk` download steps from
`unstructured`. As of `unstructured` `0.4.14`, this is handled
automatically in the relevant modules within `unstructured`.
2023-02-23 12:34:44 -08:00
Justin Torre
5bc6dc076e
added caching and properties docs (#1255) 2023-02-23 11:03:04 -08:00
Iskren Ivov Chernev
8e3cd3e0dd
Add DeepInfra LLM support (#1232)
DeepInfra is an Inference-as-a-Service provider. Add a simple wrapper
using HTTPS requests.
2023-02-23 07:37:15 -08:00
Dmitri Melikyan
b7765a95a0
docs: add Graphsignal ecosystem page (#1228)
Adds a Graphsignal ecosystem page
2023-02-23 07:33:00 -08:00
Harrison Chase
6085fe18d4
add ifttt tool (#1244) 2023-02-22 22:29:43 -08:00
Harrison Chase
71709ad5d5
Update key_concepts.md (#1209) (#1237)
Link for easier navigation (it's not immediately clear where to find
more info on SimpleSequentialChain (3 clicks away)

---------

Co-authored-by: Larry Fisherman <l4rryfisherman@protonmail.com>
2023-02-22 13:30:53 -08:00
Dennis Antela Martinez
53c67e04d4
add aleph alpha llm (#1207)
Integrate Aleph Alpha's client into Langchain to provide access to the
luminous models - more info on latest benchmarks here:
https://www.aleph-alpha.com/luminous-performance-benchmarks
2023-02-22 10:37:36 -08:00
Ikko Eltociear Ashimine
334b553260
Update petals.md (#1225)
Huggingface -> Hugging Face
2023-02-22 10:34:16 -08:00
Sason
cc7d2e5621
Correct typo in "Question Answering" How-To Guide (#1221) 2023-02-21 17:02:58 -08:00
Matt Robinson
3d5f56a8a1
docs: add quotes to unstructured[local-inference] install instructions (#1208)
### Summary

Corrects the install instruction for local inference to `pip install
"unstructured[local-inference]"`
2023-02-21 08:06:43 -08:00
Harrison Chase
047231840d
add docs for chroma persistance (#1202) 2023-02-20 23:04:17 -08:00
Harrison Chase
5bdb8dd6fe
Harrison/unstructured io (#1200) 2023-02-20 22:54:49 -08:00
Harrison Chase
d90a287d8f
Harrison/updating docs (#1196) 2023-02-20 22:54:26 -08:00
Dennis Antela Martinez
23243ae69c
add gitbook document loader (#1180)
Added a GitBook document loader. It lets you both, (1) fetch text from
any single GitBook page, or (2) fetch all relative paths and return
their respective content in Documents.

I've modified the `scrape` method in the `WebBaseLoader` to accept
custom web paths if given, but happy to remove it and move that logic
into the `GitbookLoader` itself.
2023-02-20 20:05:04 -08:00
Naveen Tatikonda
0118706fd6
Add Support for OpenSearch Vector database (#1191)
### Description
This PR adds a wrapper which adds support for the OpenSearch vector
database. Using opensearch-py client we are ingesting the embeddings of
given text into opensearch cluster using Bulk API. We can perform the
`similarity_search` on the index using the 3 popular searching methods
of OpenSearch k-NN plugin:

- `Approximate k-NN Search` use approximate nearest neighbor (ANN)
algorithms from the [nmslib](https://github.com/nmslib/nmslib),
[faiss](https://github.com/facebookresearch/faiss), and
[Lucene](https://lucene.apache.org/) libraries to power k-NN search.
- `Script Scoring` extends OpenSearch’s script scoring functionality to
execute a brute force, exact k-NN search.
- `Painless Scripting` adds the distance functions as painless
extensions that can be used in more complex combinations. Also, supports
brute force, exact k-NN search like Script Scoring.

### Issues Resolved 
https://github.com/hwchase17/langchain/issues/1054

---------

Signed-off-by: Naveen Tatikonda <navtat@amazon.com>
2023-02-20 18:39:34 -08:00
Harrison Chase
926c121b98
Harrison/text splitter docs (#1188) 2023-02-20 15:14:03 -08:00
Harrison Chase
91446a5e9b
clean up text splitting docs (#1184) 2023-02-20 11:24:31 -08:00
Harrison Chase
5a954efdd7
update gallery with slack bot (#1177) 2023-02-20 08:21:00 -08:00
9962bda70b
searx_search: docs updates (#1175)
- fix notebook formatting, remove empty cells and add scrolling for long
text

---------

Co-authored-by: blob42 <spike@w530>
2023-02-20 06:46:44 -08:00
Harrison Chase
4f3fbd7267
improve docs for indexes (#1146) 2023-02-19 23:14:50 -08:00
Harrison Chase
28781a6213
Harrison/markdown splitter (#1169)
Co-authored-by: Michael Chen <flamingdescent@gmail.com>
Co-authored-by: Michael Chen <michaelchen@stripe.com>
2023-02-19 21:31:58 -08:00
Nan Wang
e8f224fd3a
docs: add missing links to toc (#1163)
add missing links to toc

---------

Signed-off-by: Nan Wang <nan.wang@jina.ai>
2023-02-19 21:15:11 -08:00
Nick
afe884fb96
AI21 documentation incorrectly titled Cohere (#1167) 2023-02-19 21:14:59 -08:00
Harrison Chase
955c89fccb
pass in prompts to vectordbqa (#1158) 2023-02-19 20:47:17 -08:00
Harrison Chase
65cc81c479
directory loader improvements (#1162) 2023-02-19 20:47:08 -08:00
Harrison Chase
9d6d8f85da
Harrison/self hosted runhouse (#1154)
Co-authored-by: Donny Greenberg <dongreenberg2@gmail.com>
Co-authored-by: John Dagdelen <jdagdelen@users.noreply.github.com>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
Co-authored-by: Andrew White <white.d.andrew@gmail.com>
Co-authored-by: Peng Qu <82029664+pengqu123@users.noreply.github.com>
Co-authored-by: Matt Robinson <mthw.wm.robinson@gmail.com>
Co-authored-by: jeff <tangj1122@gmail.com>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MacBook-Pro.local>
Co-authored-by: zanderchase <zander@unfold.ag>
Co-authored-by: Charles Frye <cfrye59@gmail.com>
Co-authored-by: zanderchase <zanderchase@gmail.com>
Co-authored-by: Shahriar Tajbakhsh <sh.tajbakhsh@gmail.com>
Co-authored-by: Stefan Keselj <skeselj@princeton.edu>
Co-authored-by: Francisco Ingham <fpingham@gmail.com>
Co-authored-by: Dhruv Anand <105786647+dhruv-anand-aintech@users.noreply.github.com>
Co-authored-by: cragwolfe <cragcw@gmail.com>
Co-authored-by: Anton Troynikov <atroyn@users.noreply.github.com>
Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
Co-authored-by: Oliver Klingefjord <oliver@klingefjord.com>
Co-authored-by: blob42 <contact@blob42.xyz>
Co-authored-by: blob42 <spike@w530>
Co-authored-by: Enrico Shippole <henryshippole@gmail.com>
Co-authored-by: Ibis Prevedello <ibiscp@gmail.com>
Co-authored-by: jped <jonathanped@gmail.com>
Co-authored-by: Justin Torre <justintorre75@gmail.com>
Co-authored-by: Ivan Vendrov <ivan@anthropic.com>
Co-authored-by: Sasmitha Manathunga <70096033+mmz-001@users.noreply.github.com>
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
Co-authored-by: Matt Robinson <mrobinson@unstructuredai.io>
Co-authored-by: Jeff Huber <jeffchuber@gmail.com>
Co-authored-by: Akshay <64036106+akshayvkt@users.noreply.github.com>
Co-authored-by: Andrew Huang <jhuang16888@gmail.com>
Co-authored-by: rogerserper <124558887+rogerserper@users.noreply.github.com>
Co-authored-by: seanaedmiston <seane999@gmail.com>
Co-authored-by: Hasegawa Yuya <52068175+Hase-U@users.noreply.github.com>
Co-authored-by: Ivan Vendrov <ivendrov@gmail.com>
Co-authored-by: Chen Wu (吴尘) <henrychenwu@cmu.edu>
Co-authored-by: Dennis Antela Martinez <dennis.antela@gmail.com>
Co-authored-by: Maxime Vidal <max.vidal@hotmail.fr>
Co-authored-by: Rishabh Raizada <110235735+rishabh-ti@users.noreply.github.com>
2023-02-19 09:53:45 -08:00
CG80499
af8f5c1a49
Added constitutional chain. (#1147)
- Added self-critique constitutional chain based on this
[paper](https://www.anthropic.com/constitutional.pdf).
2023-02-18 19:31:51 -08:00
Harrison Chase
a83ba44efa
Harrison/ver0089 (#1144) 2023-02-18 14:25:37 -08:00
Ankush Gola
7b5e160d28
Make Tools own model, add ToolKit Concept (#1095)
Follow-up of @hinthornw's PR:

- Migrate the Tool abstraction to a separate file (`BaseTool`).
- `Tool` implementation of `BaseTool` takes in function and coroutine to
more easily maintain backwards compatibility
- Add a Toolkit abstraction that can own the generation of tools around
a shared concept or state

---------

Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
Co-authored-by: Francisco Ingham <fpingham@gmail.com>
Co-authored-by: Dhruv Anand <105786647+dhruv-anand-aintech@users.noreply.github.com>
Co-authored-by: cragwolfe <cragcw@gmail.com>
Co-authored-by: Anton Troynikov <atroyn@users.noreply.github.com>
Co-authored-by: Oliver Klingefjord <oliver@klingefjord.com>
Co-authored-by: William Fu-Hinthorn <whinthorn@Williams-MBP-3.attlocal.net>
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
2023-02-18 13:40:43 -08:00
Harrison Chase
45b5640fe5
fix sql (#1141) 2023-02-18 11:49:08 -08:00
Sam Hogan
85c1449a96
Fix typo in HyDE docs (#1142) 2023-02-18 11:48:46 -08:00
Harrison Chase
fb3c73d194
add srt loader (#1140) 2023-02-18 10:58:39 -08:00
Harrison Chase
483821ea3b
fix docs (#1133) 2023-02-18 08:13:54 -08:00
Harrison Chase
d5f3dfa1e1
Harrison/hn loader (#1130)
Co-authored-by: William X <william.y.xuan@gmail.com>
2023-02-17 15:15:02 -08:00
Harrison Chase
511d41114f
return source documents for chat vector db chain (#1128) 2023-02-17 13:40:52 -08:00
Matt Robinson
b956070f08
docs: add an unstructured section to the ecosystem page (#1125)
### Summary

Adds an Unstructured section to the ecosystem page.
2023-02-17 13:02:23 -08:00
Francisco Ingham
3462130e2d
Modify number of types of chains (#1089)
Changed number of types of chains to make it consistent with the rest of
the docs
2023-02-16 07:06:30 -08:00
Harrison Chase
7745505482
chat qa with sources (#1084) 2023-02-16 00:29:47 -08:00
Harrison Chase
badeeb37b0
fix stuff count (#1083) 2023-02-15 23:57:13 -08:00
Harrison Chase
971458c5de
docs for batch size (#1082) 2023-02-15 23:53:56 -08:00
Harrison Chase
5e10e19bfe
Harrison/align table (#1081)
Co-authored-by: Francisco Ingham <fpingham@gmail.com>
2023-02-15 23:53:37 -08:00
Harrison Chase
c60954d0f8
Harrison/telegram loader (#1080)
Co-authored-by: Maxime Vidal <max.vidal@hotmail.fr>
2023-02-15 23:24:32 -08:00
Dennis Antela Martinez
a1c296bc3c
docs: increase width (#1049)
This addresses #948.

I set the documentation max width to 2560px, but can be adjusted - see
screenshot below.

<img width="1741" alt="Screenshot 2023-02-14 at 13 05 57"
src="https://user-images.githubusercontent.com/23406704/218749076-ea51e90a-a220-4558-b4fe-5a95b39ebf15.png">
2023-02-15 23:07:01 -08:00
Harrison Chase
19c2797bed
add anthropic example (#1041)
Co-authored-by: Ivan Vendrov <ivendrov@gmail.com>
Co-authored-by: Sasmitha Manathunga <70096033+mmz-001@users.noreply.github.com>
2023-02-15 23:04:28 -08:00
3ecdea8be4
SearxNG meta search api helper (#854)
This is a work in progress PR to track my progres.

## TODO:

- [x]  Get results using the specifed searx host
- [x]  Prioritize returning an  `answer`  or results otherwise
    - [ ] expose the field `infobox` when available
    - [ ] expose `score` of result to help agent's decision
- [ ] expose the `suggestions` field to agents so they could try new
queries if no results are found with the orignial query ?

- [ ] Dynamic tool description for agents ?
- Searx offers many engines and a search syntax that agents can take
advantage of. It would be nice to generate a dynamic Tool description so
that it can be used many times as a tool but for different purposes.

- [x]  Limit number of results
- [ ]   Implement paging
- [x]  Miror the usage of the Google Search tool
- [x] easy selection of search engines
- [x]  Documentation
    - [ ] update HowTo guide notebook on Search Tools
- [ ] Handle async 
- [ ]  Tests

###  Add examples / documentation on possible uses with
 - [ ]  getting factual answers with `!wiki` option and `infoboxes`
 - [ ]  getting `suggestions`
 - [ ]  getting `corrections`

---------

Co-authored-by: blob42 <spike@w530>
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-02-15 23:03:57 -08:00
seanaedmiston
f0a258555b
Support similarity search by vector (in FAISS) (#961)
Alternate implementation to PR #960 Again - only FAISS is implemented.
If accepted can add this to other vectorstores or leave as
NotImplemented? Suggestions welcome...
2023-02-15 22:50:00 -08:00
Jonathan Pedoeem
05ad399abe
Update PromptLayerOpenAI LLM to include support for ASYNC API (#1066)
This PR updates `PromptLayerOpenAI` to now support requests using the
[Async
API](https://langchain.readthedocs.io/en/latest/modules/llms/async_llm.html)
It also updates the documentation on Async API to let users know that
PromptLayerOpenAI also supports this.

`PromptLayerOpenAI` now redefines `_agenerate` a similar was to how it
redefines `_generate`
2023-02-15 22:48:09 -08:00
Harrison Chase
98186ef180
Harrison/evernote nb (#1078)
Co-authored-by: Akshay <64036106+akshayvkt@users.noreply.github.com>
2023-02-15 22:47:30 -08:00
rogerserper
e46cd3b7db
Google Search API integration with serper.dev (wrapper, tests, docs, … (#909)
Adds Google Search integration with [Serper](https://serper.dev) a
low-cost alternative to SerpAPI (10x cheaper + generous free tier).
Includes documentation, tests and examples. Hopefully I am not missing
anything.

Developers can sign up for a free account at
[serper.dev](https://serper.dev) and obtain an api key.

## Usage

```python
from langchain.utilities import GoogleSerperAPIWrapper
from langchain.llms.openai import OpenAI
from langchain.agents import initialize_agent, Tool

import os
os.environ["SERPER_API_KEY"] = ""
os.environ['OPENAI_API_KEY'] = ""

llm = OpenAI(temperature=0)
search = GoogleSerperAPIWrapper()
tools = [
    Tool(
        name="Intermediate Answer",
        func=search.run
    )
]

self_ask_with_search = initialize_agent(tools, llm, agent="self-ask-with-search", verbose=True)
self_ask_with_search.run("What is the hometown of the reigning men's U.S. Open champion?")
```

### Output
```
Entering new AgentExecutor chain...
 Yes.
Follow up: Who is the reigning men's U.S. Open champion?
Intermediate answer: Current champions Carlos Alcaraz, 2022 men's singles champion.
Follow up: Where is Carlos Alcaraz from?
Intermediate answer: El Palmar, Spain
So the final answer is: El Palmar, Spain

> Finished chain.

'El Palmar, Spain'
```
2023-02-15 22:47:17 -08:00
Jonathan Pedoeem
05df480376
Update PromptLayerOpenAI LLM usage instructions in documentation (#1053)
This PR updates the usage instructions for PromptLayerOpenAI in
Langchain's documentation. The updated instructions provide more detail
and conform better to the style of other LLM integration documentation
pages.

No code changes were made in this PR, only improvements to the
documentation. This update will make it easier for users to understand
how to use `PromptLayerOpenAI`
2023-02-15 22:37:48 -08:00
Ankush Gola
d8ac274fc2
add to async chain notebook (#1056) 2023-02-14 18:20:38 -08:00
Ankush Gola
caa8e4742e
Enable streaming for OpenAI LLM (#986)
* Support a callback `on_llm_new_token` that users can implement when
`OpenAI.streaming` is set to `True`
2023-02-14 15:06:14 -08:00
Sasmitha Manathunga
c67c5383fd
docs: fix typo in notebook (#1046) 2023-02-14 07:06:08 -08:00
Harrison Chase
88bebb4caa
Harrison/llm integrations (#1039)
Co-authored-by: jped <jonathanped@gmail.com>
Co-authored-by: Justin Torre <justintorre75@gmail.com>
Co-authored-by: Ivan Vendrov <ivan@anthropic.com>
2023-02-13 22:06:25 -08:00
Harrison Chase
ec727bf166
Align table info (#999) (#1034)
Currently the chain is getting the column names and types on the one
side and the example rows on the other. It is easier for the llm to read
the table information if the column name and examples are shown together
so that it can easily understand to which columns do the examples refer
to. For an instantiation of this, please refer to the changes in the
`sqlite.ipynb` notebook.

Also changed `eval` for `ast.literal_eval` when interpreting the results
from the sample row query since it is a better practice.

---------

Co-authored-by: Francisco Ingham <>

---------

Co-authored-by: Francisco Ingham <fpingham@gmail.com>
2023-02-13 21:48:41 -08:00
Enrico Shippole
f30dcc6359
Add GooseAI, CerebriumAI, Petals, ForefrontAI (#981)
Add GooseAI, CerebriumAI, Petals, ForefrontAI
2023-02-13 21:20:19 -08:00
Harrison Chase
6a31a59400
add links (#1027) 2023-02-13 16:33:30 -08:00
Harrison Chase
7fb33fca47
chroma docs (#1012) 2023-02-12 23:02:01 -08:00
Harrison Chase
0c553d2064
Harrion/kg (#1016)
Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
2023-02-12 23:01:26 -08:00
cragwolfe
05d8969c79
Unstructured example notebook: add a pdf, related deps (#1011)
Updates the Unstructured example notebook with a PDF example. Includes
additional dependencies for PDF processing (and images, etc).
2023-02-12 14:56:48 -08:00
Dhruv Anand
03e5794978
typo fix on chat vector db docs (#1007)
simple typo fix: because --> between
2023-02-12 12:09:21 -08:00
Harrison Chase
0998577dfe
Harrison/unstructured structured (#1004) 2023-02-12 07:36:11 -08:00
Harrison Chase
bbb06ca4cf
pdfminer (#1003) 2023-02-12 07:29:26 -08:00
Francisco Ingham
0b6aa6a024
Added initial capital letter to bullet points that had it missing (#1000)
Co-authored-by: Francisco Ingham <>
2023-02-11 20:31:34 -08:00
Harrison Chase
10e7297306
Harrison/fake llm (#990)
Co-authored-by: Stefan Keselj <skeselj@princeton.edu>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
2023-02-11 15:12:35 -08:00
Harrison Chase
e51fad1488
Harrison/0083 (#996)
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
2023-02-11 08:29:28 -08:00
Harrison Chase
2e96704d59
Harrison/airbyte (#989)
Co-authored-by: zanderchase <zanderchase@gmail.com>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MacBook-Pro.local>
2023-02-10 18:08:00 -08:00
Charles Frye
e9799d6821
improves huggingface_hub example (#988)
The provided example uses the default `max_length` of `20` tokens, which
leads to the example generation getting cut off. 20 tokens is way too
short to show CoT reasoning, so I boosted it to `64`.

Without knowing HF's API well, it can be hard to figure out just where
those `model_kwargs` come from, and `max_length` is a super critical
one.
2023-02-10 17:56:15 -08:00
zanderchase
c2d1d903fa
Zander/online pdf loader (#984) 2023-02-10 15:42:30 -08:00
Harrison Chase
055a53c27f
add texts example (#985)
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MacBook-Pro.local>
2023-02-10 12:32:44 -08:00
jeff
6ab432d62e
docs: update spelling typos (#982)
Wonder why "with" is spelled "wiht" so many times by human
2023-02-10 11:37:59 -08:00
Matt Robinson
07a407d89a
feat: adds UnstructuredURLLoader for loading data from urls (#979)
### Summary

Adds a `UnstructuredURLLoader` that supports loading data from a list of
URLs.


### Testing

```python
from langchain.document_loaders import UnstructuredURLLoader

urls = [
    "https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-8-2023",
    "https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-9-2023"
]
loader = UnstructuredURLLoader(urls=urls)
raw_documents = loader.load()
```
2023-02-10 10:18:38 -08:00
Harrison Chase
c64f98e2bb
Harrison/format agent instructions (#973)
Co-authored-by: Andrew White <white.d.andrew@gmail.com>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
Co-authored-by: Peng Qu <82029664+pengqu123@users.noreply.github.com>
2023-02-10 10:07:26 -08:00
Harrison Chase
5469d898a9
Harrison/everynote (#974)
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
2023-02-10 08:02:35 -08:00
Harrison Chase
3d639d1539
update lint (#975)
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
2023-02-10 08:01:13 -08:00
Harrison Chase
01fa2d8117
Harrison/youtube fixes (#955)
Co-authored-by: Ji <jizhang.work@gmail.com>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
2023-02-09 08:12:22 -08:00
zanderchase
8e126bc9bd
adding webpage loading logic (#942) 2023-02-09 07:52:50 -08:00
Harrison Chase
c71027e725
add docs for steamship deployment (#949)
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
2023-02-08 16:01:19 -08:00
Harrison Chase
3e1901e1aa
gutenberg books (#946)
Co-authored-by: zanderchase <zander@unfold.ag>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
2023-02-08 12:00:47 -08:00
jeff
6a4f602156
docs: fix spelling typo (#934) 2023-02-08 11:13:35 -08:00
Ikko Eltociear Ashimine
6023d5be09
Update huggingface_hub.ipynb (#944)
HuggingFace -> Hugging Face
2023-02-08 11:05:28 -08:00
Harrison Chase
44ecec3896
Harrison/add roam loader (#939) 2023-02-08 00:35:33 -08:00
Ankush Gola
bc7e56e8df
Add asyncio support for LLM (OpenAI), Chain (LLMChain, LLMMathChain), and Agent (#841)
Supporting asyncio in langchain primitives allows for users to run them
concurrently and creates more seamless integration with
asyncio-supported frameworks (FastAPI, etc.)

Summary of changes:

**LLM**
* Add `agenerate` and `_agenerate`
* Implement in OpenAI by leveraging `client.Completions.acreate`

**Chain**
* Add `arun`, `acall`, `_acall`
* Implement them in `LLMChain` and `LLMMathChain` for now

**Agent**
* Refactor and leverage async chain and llm methods
* Add ability for `Tools` to contain async coroutine
* Implement async SerpaPI `arun`

Create demo notebook.

Open questions:
* Should all the async stuff go in separate classes? I've seen both
patterns (keeping the same class and having async and sync methods vs.
having class separation)
2023-02-07 21:21:57 -08:00
Vincent Elster
afc7f1b892
Fix typos (#929)
accomplisehd -> accomplished
2023-02-07 14:39:45 -08:00
Harrison Chase
637c0d6508
Harrison/obsidian (#920) 2023-02-06 22:21:16 -08:00
Harrison Chase
1e56879d38
Harrison/save faiss (#916)
Co-authored-by: Shrey Joshi <shreyjoshi2004@gmail.com>
2023-02-06 21:44:50 -08:00
Ankush Gola
6bd1529cb7
add GoogleDriveLoader (#914)
only deal with docs files for now
2023-02-06 21:44:35 -08:00
Harrison Chase
cc20b9425e
add reqs (#918) 2023-02-06 20:30:03 -08:00
Harrison Chase
cea380174f
fix docs custom prompt template (#917) 2023-02-06 20:29:48 -08:00
Harrison Chase
87fad8fc00
analyze document (#731)
add analyze document chain, which does text splitting and then analysis
2023-02-06 20:02:19 -08:00
Harrison Chase
e2b834e427
Harrison/prompt template prefix (#888)
Co-authored-by: Gabriel Simmons <simmons.gabe@gmail.com>
2023-02-06 19:09:28 -08:00
Harrison Chase
f95cedc443
Harrison/sql rows (#915)
Co-authored-by: Jon Luo <20971593+jzluo@users.noreply.github.com>
2023-02-06 18:56:18 -08:00
Harrison Chase
2ec25ddd4c
add unstructured examples (#913) 2023-02-06 18:13:46 -08:00
Harrison Chase
71e662e88d
update docs (#905) 2023-02-06 00:26:20 -08:00
Harrison Chase
53d56d7650
Harrison/unstructured support (#903) 2023-02-05 23:02:07 -08:00
Harrison Chase
2a68be3e8d
chat vector db chain (#902) 2023-02-05 21:38:47 -08:00
James Briggs
8217a2f26c
Update pinecone init details in docs (#898)
PR to fix outdated environment details in the docs, see issue #897 

I added code comments as pointers to where users go to get API keys, and
where they can find the relevant environment variable.
2023-02-05 15:21:56 -08:00
Harrison Chase
a2b699dcd2
prompt template from string (#884) 2023-02-04 17:04:58 -08:00
Alex
7cc44b3bdb
Add to gallery (#882) 2023-02-04 09:45:20 -08:00
Harrison Chase
0b9f086d36
Harrison/docs splitter (#879) 2023-02-03 15:09:13 -08:00
Ryan Walker
1dd0733515
Fix small typo in getting started docs (#876)
Just noticed this little typo while reading the docs, thought I'd open a
PR!
2023-02-03 14:22:12 -08:00
Zach Schillaci
4c79100b15
Correct prompt typo + update example for SQLDatabaseChain (#868)
See https://github.com/hwchase17/langchain/issues/821
2023-02-03 08:34:41 -08:00
Harrison Chase
3f48eed5bd
Harrison/milvus (#856)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
Signed-off-by: Frank Liu <frank.liu@zilliz.com>
Co-authored-by: Filip Haltmayer <81822489+filip-halt@users.noreply.github.com>
Co-authored-by: Frank Liu <frank@frankzliu.com>
2023-02-02 22:05:47 -08:00
Harrison Chase
523ad2e6bd
vercel deployments (#850) 2023-02-02 19:54:09 -08:00
Harrison Chase
fc0cfd7d1f
docs (#848) 2023-02-02 11:35:36 -08:00
Harrison Chase
23d5f64bda
Harrison/ngram example (#846)
Co-authored-by: Sean Spriggens <ssprigge@syr.edu>
2023-02-02 09:44:42 -08:00
Harrison Chase
0de55048b7
return code for pal (#844) 2023-02-02 08:47:20 -08:00
Harrison Chase
d564308e0f
rfc: instruct embeddings (#811)
Co-authored-by: seanaedmiston <seane999@gmail.com>
2023-02-02 08:44:02 -08:00
Eli Mernit
bfabd1d5c0
Added new deployment template (#835)
This PR introduces a new template for deploying LangChain apps as web
endpoints. It includes template code, and links to a detailed
code-walkthrough.
2023-02-01 23:38:36 -08:00
Istora Mandiri
06438794e1
Fix typo in textsplitter docs (#825) 2023-02-01 23:32:35 -08:00
Harrison Chase
b0d560be56
add to gallery (#824) 2023-02-01 07:10:15 -08:00
Harrison Chase
7b4882a2f4
Harrison/tf embeddings (#817)
Co-authored-by: Ryohei Kuroki <10434946+yakigac@users.noreply.github.com>
2023-01-31 00:00:08 -08:00
Harrison Chase
94ae126747
return sql intermediate steps (#792) 2023-01-30 15:10:48 -08:00
Roy Williams
6086292252
Centralize logic for loading from LangChainHub, add ability to pin dependencies (#805)
It's generally considered to be a good practice to pin dependencies to
prevent surprise breakages when a new version of a dependency is
released. This commit adds the ability to pin dependencies when loading
from LangChainHub.

Centralizing this logic and using urllib fixes an issue identified by
some windows users highlighted in this video -
https://youtu.be/aJ6IQUh8MLQ?t=537
2023-01-30 14:52:17 -08:00
Harrison Chase
7728a848d0
Harrison/tracing docs (#806)
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
2023-01-29 20:49:35 -08:00
Harrison Chase
f3da4dc6ba
Harrison/tracing docs (#804)
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
2023-01-29 20:24:22 -08:00
Harrison Chase
ae1b589f60
Harrison/add link for support (#794) 2023-01-28 22:53:04 -08:00
Harrison Chase
1ad7973cc6
Harrison/tool decorator (#790)
Co-authored-by: Jason Liu <jxnl@users.noreply.github.com>
Co-authored-by: Jason Liu <jason@jxnl.coA>
2023-01-28 18:26:24 -08:00
Harrison Chase
248c297f1b
Sample row in table info for SQLDatabase (#769) (#782)
The agents usually benefit from understanding what the data looks like
to be able to filter effectively. Sending just one row in the table info
allows the agent to understand the data before querying and get better
results.

---------

Co-authored-by: Francisco Ingham <>

---------

Co-authored-by: Francisco Ingham <fpingham@gmail.com>
2023-01-28 13:37:07 -08:00
Harrison Chase
c658f0aed3
Harrison/add to search (#778)
Co-authored-by: Enrico Shippole <enricoship@gmail.com>
2023-01-28 08:06:00 -08:00
Harrison Chase
a5d003f0c9
update notebook and make backwards compatible (#772) 2023-01-28 07:23:04 -08:00
Harrison Chase
b9ad214801
add docs for loading from hub (#763) 2023-01-27 07:10:26 -08:00
Harrison Chase
1b89a438cf
(wip) Harrison/serialize agents (#725) 2023-01-26 19:48:47 -08:00
Roy Williams
d2f882158f
Add type information for crawler.py (#738)
Added type information to `crawler.py` to make it safer to use and
understand.
2023-01-26 19:37:31 -08:00
Harrison Chase
bd0bf4e0a9
Harrison/generate blog post (#732)
Co-authored-by: Ren <yirenlu92@users.noreply.github.com>
2023-01-24 22:54:12 -08:00
scadEfUr
e3df8ab6dc
move hyde into chains (#728)
Co-authored-by: scadEfUr <>
2023-01-24 22:23:32 -08:00
Harrison Chase
0ffeabd14f
Harrison/serialize llm chain (#671) 2023-01-24 21:36:19 -08:00
Sam Hogan
499e54edda
fix typos in readme and text splitter docs (#720)
Fix typos in readme and TextSplitter documentation.
2023-01-24 10:59:23 -08:00
Николай Шангин
18b1466893
Fix not imported 'validator' (#715)
otherwise `@validator("input_variables")` do not work
2023-01-24 07:06:50 -08:00
Harrison Chase
b69b551c8b
clarify use cases (#711) 2023-01-24 00:37:26 -08:00
Nicolas
66fd57878a
docs: Update vector_db_qa_with_sources.ipynb (#706) 2023-01-23 23:06:54 -08:00
Harrison Chase
fc4ad2db0f
langchain hub docs (#704)
Co-authored-by: scadEfUr <123224380+scadEfUr@users.noreply.github.com>
2023-01-23 23:06:23 -08:00
Harrison Chase
3a30e6daa8
Harrison/openai callback (#684) 2023-01-22 23:37:01 -08:00
Amos Ng
8baf6fb920
Update examples to fix execution problems (#685)
On the [Getting Started
page](https://langchain.readthedocs.io/en/latest/modules/prompts/getting_started.html)
for prompt templates, I believe the very last example

```python
print(dynamic_prompt.format(adjective=long_string))
```

should actually be

```python
print(dynamic_prompt.format(input=long_string))
```

The existing example produces `KeyError: 'input'` as expected

***

On the [Create a custom prompt
template](https://langchain.readthedocs.io/en/latest/modules/prompts/examples/custom_prompt_template.html#id1)
page, I believe the line

```python
Function Name: {kwargs["function_name"]}
```

should actually be

```python
Function Name: {kwargs["function_name"].__name__}
```

The existing example produces the prompt:

```
        Given the function name and source code, generate an English language explanation of the function.
        Function Name: <function get_source_code at 0x7f907bc0e0e0>
        Source Code:
        def get_source_code(function_name):
    # Get the source code of the function
    return inspect.getsource(function_name)

        Explanation:
```

***

On the [Example
Selectors](https://langchain.readthedocs.io/en/latest/modules/prompts/examples/example_selectors.html)
page, the first example does not define `example_prompt`, which is also
subtly different from previous example prompts used. For user
convenience, I suggest including

```python
example_prompt = PromptTemplate(
    input_variables=["input", "output"],
    template="Input: {input}\nOutput: {output}",
)
```

in the code to be copy-pasted
2023-01-22 14:49:25 -08:00
Nicolas
4ddfa82bb7
docs: small typo on serpapi.md (#693) 2023-01-22 13:10:24 -08:00
Nicolas
34cb8850e9
docs: small typo google_search.md (#692) 2023-01-22 13:09:15 -08:00
Samantha Whitmore
77e3d58922
ConversationEntityMemory: Chain which uses an entity extraction & sum… (#678)
…marization prompt to maintain a key-value store of memory information

cc @devennavani

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-01-22 10:10:02 -08:00
Ikko Eltociear Ashimine
64580259d0
Fix typo in hyde.ipynb (#688)
therefor -> therefore
2023-01-22 08:21:31 -08:00
Harrison Chase
e45f7e40e8
Harrison/few shot yaml (#682)
Co-authored-by: vintro <77507980+vintrocode@users.noreply.github.com>
2023-01-21 16:08:03 -08:00
Will Olson
2f57d18b25
Update hyperlink in Custom Prompt Template page (#677)
The current link points to a non-existent page. I've updated the link to
match what is on the "Create a custom example selector" page.

<img width="584" alt="Screen Shot 2023-01-21 at 10 33 05 AM"
src="https://user-images.githubusercontent.com/6773706/213879535-d8f2953d-ac37-448d-9b32-fdeb7b73cc32.png">
2023-01-21 16:03:21 -08:00
Harrison Chase
3d41af0aba
Harrison/load tools kwargs (#681)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
2023-01-21 16:03:02 -08:00
Harrison Chase
0b204d8c21
Harrison/quadrant (#665)
Co-authored-by: Kacper Łukawski <kacperlukawski@users.noreply.github.com>
2023-01-20 09:45:01 -08:00
Harrison Chase
d0fdc6da11
Harrison/bing wrapper (#656)
Co-authored-by: Enrico Shippole <henryshippole@gmail.com>
2023-01-19 14:48:30 -08:00
Charles Frye
bfb23f4608
typo bugfixes in getting started with prompts (#651)
tl;dr: input -> word, output -> antonym, rename to dynamic_prompt
consistently

The provided code in this example doesn't run, because the keys are
`word` and `antonym`, rather than `input` and `output`.

Also, the `ExampleSelector`-based prompt is named `few_shot_prompt` when
defined and `dynamic_prompt` in the follow-up example. The former name
is less descriptive and collides with an earlier example, so I opted for
the latter.

Thanks for making a really cool library!
2023-01-19 07:05:20 -08:00
John
3adc5227cd
typo (#650) 2023-01-19 07:03:11 -08:00
Harrison Chase
30abfc41c2
add instructions for saving loading (#642) 2023-01-18 00:19:05 -08:00
Harrison Chase
95720adff5
Add documentation for custom prompts for Agents (#631) (#640)
- Added a comment interpreting regex for `ZeroShotAgent`
- Added a note to the `Custom Agent` notebook

Co-authored-by: Sam Ching <samuel@duolingo.com>
2023-01-17 22:47:15 -08:00
Harrison Chase
6be5f4e4c4
Harrison/sql db chain (#641)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
2023-01-17 22:32:28 -08:00
Chetanya Rastogi
b550f57912
Fix the env variable for OpenAI Base Url (#639)
For using Azure OpenAI API, we need to set multiple env vars. But as can
be seen in openai package
[here](48b69293a3/openai/__init__.py (L35)),
the env var for setting base url is named `OPENAI_API_BASE` and not
`OPENAI_API_BASE_URL`. This PR fixes that part in the documentation.
2023-01-17 22:30:29 -08:00
Francis
b374d481c8
fix typo (#636)
there is a small typo in one of the docs.
2023-01-17 22:17:50 -08:00
Harrison Chase
3d43906572
Harrison/new api chain (#623)
Co-authored-by: Francisco Ingham <fpingham@gmail.com>
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>
2023-01-15 18:34:43 -08:00
Harrison Chase
1c71fadfdc
more complex sql chain (#619)
add a more complex sql chain that first subsets the necessary tables
2023-01-15 17:07:21 -08:00
Harrison Chase
2a54e73fec
bump version to 0063 (#616) 2023-01-14 08:09:25 -08:00
Harrison Chase
57bbc5d6da
improve css (#615) 2023-01-14 07:39:29 -08:00
Nicolas
91d7fd20ae
feat: add custom prompt for QAEvalChain chain (#610)
I originally had only modified the `from_llm` to include the prompt but
I realized that if the prompt keys used on the custom prompt didn't
match the default prompt, it wouldn't work because of how `apply` works.

So I made some changes to the evaluate method to check if the prompt is
the default and if not, it will check if the input keys are the same as
the prompt key and update the inputs appropriately.

Let me know if there is a better way to do this.

Also added the custom prompt to the QA eval notebook.
2023-01-14 07:23:48 -08:00
Francisco Ingham
1787c473b8
Custom prompt option for llm_bash and api chains (#612)
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>
2023-01-14 07:22:52 -08:00
Nicolas
b7225fd010
docs: fix small typo (#611) 2023-01-13 17:31:33 -08:00
Harrison Chase
9f9afbb6a8
add custom prompt for LLMMathChain and SQLDatabase chain (#605) 2023-01-13 06:28:51 -08:00
Sasmitha Manathunga
3e55f1474e
docs: fix typo (#604) 2023-01-12 21:36:03 -08:00
Sam Ching
c4c6bf6e6e
Add subsection for colab notebooks (#599)
Motivation is that these don't get lost in the Twitterverse!
2023-01-12 18:16:55 -08:00
Rukmal Weerawarana
0f544a8811
Fix minor error in LLM documentation (#602) 2023-01-12 18:16:32 -08:00
Ikko Eltociear Ashimine
60dfe58325
Fix typo in vector_db_qa.ipynb (#597)
paramter -> parameter
2023-01-12 08:23:24 -08:00
Harrison Chase
950a81399a
bump version to 61 (#596) 2023-01-12 07:20:16 -08:00
Harrison Chase
d574bf0a27
add documentation on how to load different chain types (#595) 2023-01-12 06:47:38 -08:00
Harrison Chase
956416c150
Harrison/update links1 (#594)
update links to be relative

Co-authored-by: Marc Green <marcgreen@users.noreply.github.com>
2023-01-12 06:29:42 -08:00
Harrison Chase
8ab09c18a1
Return source documents option in VectorDBQA (#585) (#592)
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>

Co-authored-by: Francisco Ingham <fpingham@gmail.com>
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>
2023-01-12 06:09:32 -08:00
Harrison Chase
7b6e7f6e12
bump to version 60 (#583) 2023-01-11 07:09:30 -08:00
Harrison Chase
f74ce7a104
Harrison/combine memories (#582)
Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
Co-authored-by: Diwank Singh Tomer <diwank.singh@gmail.com>
2023-01-11 06:08:58 -08:00
Harrison Chase
ffc7e04d44
Harrison/wolfram alpha (#579)
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
2023-01-11 05:52:19 -08:00
Harrison Chase
94765e7487
more gallery (#577) 2023-01-10 08:24:00 -08:00
Harrison Chase
50a49eff15
gallery updates (#573) 2023-01-10 07:41:29 -08:00
Harrison Chase
6966863d7d
Harrison/deployments (#572) 2023-01-10 07:41:16 -08:00
Harrison Chase
7de5139750
add example selector docs (#564) 2023-01-09 19:17:29 -08:00
Harrison Chase
b06a2a6191
improve documentation on how to pass in custom prompts (#561) 2023-01-08 19:20:13 -08:00
Harrison Chase
1192cc0767
smart text splitter (#530)
smart text splitter that iteratively tries different separators until it
works!
2023-01-08 15:11:10 -08:00
Harrison Chase
8dfad874a2
map rerank chain (#516)
add a chain that applies a prompt to all inputs and then returns not
only an answer but scores it

add examples for question answering and question answering with sources
2023-01-08 06:49:22 -08:00
Nicolas
948eee9fe1
Docs: side menu to match the order (llms) (#557)
Small quick fix:

Suggest making the order of the menu the same as it is written on the
page (Getting Started -> Key Concepts). Before the menu order was not
the same as it was on the page. Not sure if this is the only place the
menu is affected.

Mismatch is found here:
https://langchain.readthedocs.io/en/latest/modules/llms.html
2023-01-06 09:34:08 -08:00
Harrison Chase
823a44ef80
bump to 0058 (#556) 2023-01-06 07:58:38 -08:00
Harrison Chase
74932f2516
RFC: conversational agent (#464)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
2023-01-06 07:25:55 -08:00
Harrison Chase
e64ed7b975
Harrison/tools priority (#554)
Co-authored-by: Yong723 <50616781+Yongtae723@users.noreply.github.com>
2023-01-06 06:56:11 -08:00
Harrison Chase
4974f49bb7
add return_direct flag to tool (#537)
adds a return_direct flag to tools, which just returns the tool output
as the final output
2023-01-06 06:40:32 -08:00
Harrison Chase
9753bccc71
Feature: linkcheck-action (#534) (#542)
- Add support for local build and linkchecking of docs
- Add GitHub Action to automatically check links before prior to
publication
- Minor reformat of Contributing readme
- Fix existing broken links

Co-authored-by: Hunter Gerlach <hunter@huntergerlach.com>

Co-authored-by: Hunter Gerlach <HunterGerlach@users.noreply.github.com>
Co-authored-by: Hunter Gerlach <hunter@huntergerlach.com>
2023-01-04 21:39:50 -08:00
Harrison Chase
73f7ebd9d1
Harrison/sqlalchemy cache store (#536)
Co-authored-by: Jason Gill <jasongill@gmail.com>
2023-01-04 18:38:15 -08:00
Rubens Mau
020e73017b
Updated embeddings.ipynb (#531)
updated embeddings.ipynb
2023-01-04 10:43:52 -08:00
Ikko Eltociear Ashimine
ca9aaac36e
Fix typo in key_concepts.md (#535)
therefor -> therefore
2023-01-04 10:43:02 -08:00
Harrison Chase
9e04c34e20
Add BaseCallbackHandler and CallbackManager (#478)
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
2023-01-04 07:54:25 -08:00
Nuno Campos
6d78be0c83
Add link to gihub repo in header of new docs (#524) 2023-01-03 10:16:59 -08:00
Harrison Chase
0db05b6725
Harrison/add human prefix (#520)
Co-authored-by: Andrew Huang <jhuang16888@gmail.com>
2023-01-03 08:03:50 -08:00
Harrison Chase
03f185bcd5
more robust handling for max iterations (#514)
add a `generate` method which makes one final forward pass through the
llm
2023-01-03 07:46:08 -08:00
Harrison Chase
40326c698c
unify argument name (#513)
unify names in map reduce and refine chains to just be
return_intermediate_steps

also unify the return key
2023-01-03 07:45:08 -08:00
lewtun
12108104c9
Add links to Hugging Face Hub docs (#518)
This PR adds some tweaks to the Hugging Face docs, mostly with links to
the Hub + relevant docs.
2023-01-03 07:43:57 -08:00
Harrison Chase
3efec55f93
update lobby link (#517) 2023-01-02 20:25:49 -08:00
Hunter Gerlach
7253fada0d
Fix/broken getting started link (#511)
I noticed (after publication) that the getting_started link on the main
page was borked. This should fix it.

Co-authored-by: Hunter Gerlach <hunter@huntergerlach.com>
2023-01-02 10:15:17 -08:00
Harrison Chase
985496f4be
Docs refactor (#480)
Big docs refactor! Motivation is to make it easier for people to find
resources they are looking for. To accomplish this, there are now three
main sections:

- Getting Started: steps for getting started, walking through most core
functionality
- Modules: these are different modules of functionality that langchain
provides. Each part here has a "getting started", "how to", "key
concepts" and "reference" section (except in a few select cases where it
didnt easily fit).
- Use Cases: this is to separate use cases (like summarization, question
answering, evaluation, etc) from the modules, and provide a different
entry point to the code base.

There is also a full reference section, as well as extra resources
(glossary, gallery, etc)

Co-authored-by: Shreya Rajpal <ShreyaR@users.noreply.github.com>
2023-01-02 08:24:09 -08:00
Harrison Chase
d95b39d37f
version 0.0.53 (#497) 2022-12-30 11:05:18 -05:00
Harrison Chase
0072686aab
Harrison/new search engine (#477)
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
2022-12-30 08:06:57 -05:00
Shuchang Zhou
12aa43469f
Update prompt_management.ipynb (#484) 2022-12-29 21:34:32 -05:00
Harrison Chase
d0f194de73
add logic for agent stopping (#420) 2022-12-29 08:21:11 -05:00
Harrison Chase
2b84e5cda3
Harrison/fix memory and serp (#457)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
2022-12-28 11:07:57 -05:00
Harrison Chase
d98607408b
Harrison/v0050 (#452) 2022-12-28 09:22:43 -05:00
Harrison Chase
55007e71be
add output key for memory (#443)
this allows chains that return multiple values to use memory
2022-12-28 09:04:15 -05:00
Harrison Chase
5208bb8c36
make tools editable (#445)
use dataclass instead of namedtuple, which makes it editable

add example in notebook
2022-12-28 09:03:16 -05:00
Harrison Chase
90e8ccc898
Harrison/update links (#450)
Co-authored-by: Sam Ching <samuelcwl@gmail.com>
Co-authored-by: Ikko Ashimine <eltociear@gmail.com>
2022-12-28 09:02:07 -05:00
Harrison Chase
0c5d3fd894
version 0.0.49 (#436) 2022-12-27 09:17:01 -05:00
Harrison Chase
b7566b5ec3
Harrison/return intermediate steps (#428) 2022-12-27 08:22:48 -05:00
Harrison Chase
7fc4b4b3e1
Harrison/ver 0048 (#429) 2022-12-26 11:36:49 -05:00
Harrison Chase
b50a56830d
Harrison/evaluation notebook (#426) 2022-12-26 09:16:37 -05:00
Harrison Chase
97f4000d3a
fix react docstore (#427) 2022-12-26 08:46:38 -05:00
Ikko Ashimine
9ae1d75318
Update integrations.md (#424)
HuggingFace -> Hugging Face
2022-12-25 23:03:05 -05:00
Harrison Chase
0d7aa1ee99
Harrison/docs to index (#419)
Add method for going directly from documents to VectorStores

Update notebook to showcase this functionality
2022-12-25 09:53:07 -05:00
Harrison Chase
48ae981d69
Harrison/multi input tools (#421)
add documentation on how to use tools that require multiple inputs
2022-12-25 09:52:48 -05:00
Andrew Wang
4416dc9d5d
Update prompt_serialization.ipynb (#417)
Fix typo.
Originally "support methods are..."
Now "support methods *that* are.."
2022-12-24 17:53:11 -05:00
Harrison Chase
20959d8c36
check memory variables (#411)
can have multiple input keys, if some come from memory
2022-12-24 08:35:46 -05:00
altryne
f990395211
Readme typos (#409)
I was honored by the twitter mention, so used PyCharm to try and... help
docs even a little bit.
Mostly typo-s and correct spellings. 

PyCharm really complains about "really good" being used all the time and
recommended alternative wordings haha
2022-12-23 13:13:07 -05:00
Dheeraj Agrawal
ea3da9a469
Fix documentation error langchain explanation of combine_docs.md (#404)
This PR is regarding the issue here -
https://github.com/hwchase17/langchain/issues/403
2022-12-23 08:54:26 -05:00
Harrison Chase
77e1743341
update example (#402) 2022-12-22 17:09:47 -05:00
Samantha Whitmore
6bc8ae63ef
Add Redis cache implementation (#397)
I'm using a hash function for the key just to make sure its length
doesn't get out of hand, otherwise the implementation is quite similar.
2022-12-22 12:31:27 -05:00
Harrison Chase
ff03242fa0
Harrison/ver 044 (#400) 2022-12-22 11:20:18 -05:00
Harrison Chase
6b60c509ac
(WIP) add HyDE (#393)
Co-authored-by: cameronccohen <cameron.c.cohen@gmail.com>
Co-authored-by: Cameron Cohen <cameron.cohen@quantco.com>
2022-12-21 20:46:41 -05:00
Keiji Kanazawa
543db9c2df
Add Azure OpenAI LLM (#395)
Hi!  This PR adds support for the Azure OpenAI service to LangChain.

I've tried to follow the contributing guidelines.

Co-authored-by: Keiji Kanazawa <{ID}+{username}@users.noreply.github.com>
2022-12-21 20:45:37 -05:00
Harrison Chase
c104d507bf
Harrison/improve data augmented generation docs (#390)
Co-authored-by: cameronccohen <cameron.c.cohen@gmail.com>
Co-authored-by: Cameron Cohen <cameron.cohen@quantco.com>
2022-12-20 22:24:08 -05:00
Harrison Chase
ad4414b59f
update docs (#389) 2022-12-20 09:32:10 -05:00
Harrison Chase
c8b4b54479
bump version to 0.0.42 (#388) 2022-12-19 20:59:34 -05:00
Harrison Chase
47ba34c83a
split up and improve agent docs (#387) 2022-12-19 20:32:45 -05:00
Abi Raja
467aa0cee0
Fix typo in docs (#386) 2022-12-19 17:39:44 -05:00
Harrison Chase
6be5747466
RFC: add cache override to LLM class (#379) 2022-12-19 17:36:14 -05:00
Harrison Chase
46c428234f
MMR example selector (#377)
implement max marginal relevance example selector
2022-12-19 17:09:27 -05:00
Harrison Chase
ffed5e0056
Harrison/jinja formatter (#385)
Co-authored-by: Benjamin <BenderV@users.noreply.github.com>
2022-12-19 16:40:39 -05:00
Harrison Chase
a01d3e6955
fix agent memory docs (#382) 2022-12-19 09:15:32 -05:00
Harrison Chase
cf98f219f9
Harrison/tools exp (#372) 2022-12-18 21:51:23 -05:00
Harrison Chase
2eef76ed3f
fix documentation (#365) 2022-12-16 16:48:54 -08:00
Benjamin
85c1bd2cd0
add sqlalchemy generic cache (#361)
Created a generic SQLAlchemyCache class to plug any database supported
by SQAlchemy. (I am using Postgres).
I also based the class SQLiteCache class on this class SQLAlchemyCache.

As a side note, I'm questioning the need for two distinct class
LLMCache, FullLLMCache. Shouldn't we merge both ?
2022-12-16 16:47:23 -08:00
Harrison Chase
809a9f485f
Harrison/new version (#362) 2022-12-16 07:42:31 -08:00
Harrison Chase
2dd895d98c
add openai tokenizer (#355) 2022-12-15 22:35:42 -08:00
Harrison Chase
c1b50b7b13
Harrison/map reduce merge (#344)
Co-authored-by: John Nay <JohnNay@users.noreply.github.com>
2022-12-15 17:49:14 -08:00
Harrison Chase
78b31e5966
Harrison/cache (#343) 2022-12-15 07:53:32 -08:00
Harrison Chase
5161ae7e08
add new example (#345) 2022-12-14 22:31:34 -08:00
Harrison Chase
e26b6f9c89
fix batching (#339) 2022-12-14 08:25:37 -08:00
Harrison Chase
996b5a3dfb
Harrison/llm final stuff (#332) 2022-12-13 07:50:46 -08:00
Ankush Gola
8fdcdf4c2f
add .idea files to gitignore, add zsh note to installation docs (#329) 2022-12-13 05:20:22 -08:00
Harrison Chase
e02d6b2288
beta: logger (#307) 2022-12-10 23:17:19 -08:00
Harrison Chase
36b4c58acf
expose more stuff (#306) 2022-12-10 23:16:32 -08:00
Hunter Gerlach
9ee6115deb
Minor grammar fixes for memory docs to improve readability (#303)
Nothing of substance was changed. I simply corrected a few minor errors
that could slow down the reader.

Co-authored-by: Hunter Gerlach <hunter@huntergerlach.com>
2022-12-10 16:18:01 -08:00
Harrison Chase
9d08384d5f
Harrison/bump version (#300) 2022-12-10 09:37:42 -08:00
Harrison Chase
853894dd47
add moderation chain (#299) 2022-12-10 09:19:16 -08:00
andersenchen
5267ebce2d
Add LLMCheckerChain (#281)
Implementation of https://github.com/jagilley/fact-checker. Works pretty
well.

<img width="993" alt="Screenshot 2022-12-07 at 4 41 47 PM"
src="https://user-images.githubusercontent.com/101075607/206302751-356a19ff-d000-4798-9aee-9c38b7f532b9.png">

Verifying this manually:
1. "Only two kinds of egg-laying mammals are left on the planet
today—the duck-billed platypus and the echidna, or spiny anteater."
https://www.scientificamerican.com/article/extreme-monotremes/
2. "An [Echidna] egg weighs 1.5 to 2 grams (0.05 to 0.07
oz)[[19]](https://en.wikipedia.org/wiki/Echidna#cite_note-19) and is
about 1.4 centimetres (0.55 in) long."
https://en.wikipedia.org/wiki/Echidna#:~:text=sleep%20is%20suppressed.-,Reproduction,a%20reptile%2Dlike%20egg%20tooth.
3. "A [platypus] lays one to three (usually two) small, leathery eggs
(similar to those of reptiles), about 11 mm (7⁄16 in) in diameter and
slightly rounder than bird eggs."
https://en.wikipedia.org/wiki/Platypus#:~:text=It%20lays%20one%20to%20three,slightly%20rounder%20than%20bird%20eggs.
4. Therefore, an Echidna is the mammal that lays the biggest eggs.


cc @hwchase17
2022-12-09 12:49:05 -08:00
Harrison Chase
43c9bd869f
add memprompt docs (#294) 2022-12-09 12:40:24 -08:00
Ben
0f399350f1
Fix typo in Getting Started / LLM Chains docs (#291)
I noticed this typo when reading the getting started guide, hope this
fix makes sense.
2022-12-09 06:48:02 -08:00
Samantha Whitmore
b10be842f6
ChatGPT Clone: adding ConversationBufferWindowMemory to replicate vir… (#288)
…tual env example
2022-12-08 23:01:08 -08:00
Harrison Chase
e2e501aa06
Harrison/version 0032 (#283) 2022-12-08 07:59:58 -08:00
Harrison Chase
e9b1c8cdfa
Harrison/base combine doc chain (#264) 2022-12-07 22:56:26 -08:00
Harrison Chase
c27a6fa8a4
update docs (#278) 2022-12-07 08:40:08 -08:00
Harrison Chase
834b391792
update notebooks (#275) 2022-12-06 22:55:27 -08:00
coyotespike
b7bef36ee1
BashChain (#260)
Love the project, a ton of fun!

I think the PR is pretty self-explanatory, happy to make any changes! I
am working on using it in an `LLMBashChain` and may update as that
progresses.

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2022-12-06 21:57:50 -08:00
Harrison Chase
28be37f470
LLMRequestsChain (#267) 2022-12-06 21:55:02 -08:00
Harrison Chase
5cd6956d58
Harrison/version 0028 (#259) 2022-12-04 17:44:40 -08:00
Harrison Chase
b5d8434a50
Harrison/improve chain docs (#251) 2022-12-03 13:28:50 -08:00
Scott Leibrand
b4762dfff0
Refine Olivia Wilde's boyfriend example prompt to work better (#248)
With the original prompt, the chain keeps trying to jump straight to
doing math directly, without first looking up ages. With this two-part
question, it behaves more as intended:


> Entering new ZeroShotAgent chain...
How old is Olivia Wilde's boyfriend? What is that number raised to the
0.23 power?
Thought: I need to find out how old Olivia Wilde's boyfriend is, and
then use a calculator to calculate the power.
Action: Search
Action Input: Olivia Wilde's boyfriend age
Observation: While Wilde, 37, and Styles, 27, have both kept a low
profile when it comes to talking about their relationship, Wilde did
address their ...
Thought: Olivia Wilde's boyfriend is 27 years old.
Action: Calculator
Action Input: 27^0.23

> Entering new LLMMathChain chain...
27^0.23

```python
import math
print(math.pow(27, 0.23))
```

Answer: 2.1340945944237553

> Finished LLMMathChain chain.

Observation: Answer: 2.1340945944237553

Thought: I now know the final answer.
Final Answer: 2.1340945944237553
> Finished ZeroShotAgent chain.
2022-12-03 08:11:38 -08:00
Harrison Chase
024c3e1dbe
add react text world doc (#245) 2022-12-02 09:07:21 -08:00
Harrison Chase
347fc49d4d
Harrison/combine documents chain (#212)
combine documents chain powering vector db qa with sources chain
2022-11-30 22:00:02 -08:00
Harrison Chase
3bda0019ae
Harrison/list of examples (#218) 2022-11-29 20:08:00 -08:00
Harrison Chase
ca2394028f
move search to not be a chain (#226) 2022-11-29 20:07:44 -08:00
Harrison Chase
b19a73be26
pal chain touch ups (#225)
expose PAL in main entrypoint
2022-11-29 18:13:21 -08:00
Harrison Chase
1b9b8efbc9
pal chain (#207)
from https://arxiv.org/pdf/2211.10435.pdf
2022-11-28 21:38:34 -08:00
Harrison Chase
03c7140228
fix self ask template (#216) 2022-11-28 17:27:26 -08:00
Harrison Chase
d4e6b7a692
Harrison/update docs mem (#201) 2022-11-26 06:38:49 -08:00
Harrison Chase
05c5d0b8ee
add custom prompt notebooks (#198) 2022-11-26 06:07:02 -08:00
Harrison Chase
fcb9b2ffe5
Harrison/agent memory (#197)
add doc for agent with memory
2022-11-26 06:06:44 -08:00
Harrison Chase
6eab5254e5
add docs for custom agents (#196) 2022-11-26 06:03:08 -08:00
Harrison Chase
08deed9002
Harrison/memory docs (#195)
update memory docs and change variables
2022-11-26 05:58:54 -08:00
Harrison Chase
f18a08f58d
add memory to llm chain notebook (#193) 2022-11-25 18:28:55 -08:00
Harrison Chase
c3ad99a34f
Harrison/more memory docs (#192) 2022-11-25 13:00:12 -05:00
Harrison Chase
b0feb3608b
documentation (#191) 2022-11-25 12:41:27 -05:00
Samantha Whitmore
a408ed3ea3
Samantha/add conversation chain (#166)
Add MemoryChain and ConversationChain as chains that take a docstore in
addition to the prompt, and use the docstore to stuff context into the
prompt. This can be used to have an ongoing conversation with a chatbot.

Probably needs a bit of refactoring for code quality

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2022-11-23 16:35:38 -08:00
Samantha Whitmore
09f301cd38
Add add_example method to all ExampleSelector classes, with tests (#178)
Also updated docs, and noticed an issue with the add_texts method on
VectorStores that I had missed before -- the metadatas arg should be
required to match the classmethod which initializes the VectorStores
(the add_example methods break otherwise in the ExampleSelectors)
2022-11-23 13:12:47 -08:00
Harrison Chase
780ef84cf0
use action verb in documentation (#175) 2022-11-22 21:04:26 -08:00
Harrison Chase
5d887970f6
change to agent (#173) 2022-11-22 18:02:20 -08:00
Harrison Chase
d3a7429f61
(WIP) agents (#171) 2022-11-22 06:16:26 -08:00
Harrison Chase
4a4dfbfbed
Harrison/sequential chains (#168)
add support for basic sequential chains
2022-11-21 13:08:53 -08:00
Jim Salmons
e9baf9c134
Update llm.md (#164)
Without the print on the `llm` call, the new user sees no visible effect
when just getting started. The assumption here is the new user is
running this in a new sandbox script file or repl via copy-paste.
2022-11-20 15:22:53 -08:00
Harrison Chase
e49fc51492
Harrison/update docs (#162)
minor update to docs re imports
2022-11-20 07:18:43 -08:00
Harrison Chase
243211a5ae
bump version to 0017 (#161) 2022-11-20 07:04:09 -08:00
Harrison Chase
a19ad935b3
Harrison/verbose prompt (#159)
Add printing of prompt to LLMChain
2022-11-19 20:39:35 -08:00
Harrison Chase
c02eb199b6
add few shot example (#148) 2022-11-19 20:32:45 -08:00
Harrison Chase
b15c84e19d
Harrison/chain lab (#156) 2022-11-18 05:50:02 -08:00
Harrison Chase
b1b6b27c5f
Harrison/redo docs (#130)
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2022-11-13 20:13:23 -08:00
Harrison Chase
5e76c12455
Harrison/fix docs (#115) 2022-11-10 08:59:51 -08:00
Harrison Chase
a7d14cad00
add link to socratic models (#69) 2022-11-06 14:10:26 -08:00
Harrison Chase
618611f4dd
update glossary (#63) 2022-11-05 08:44:37 -07:00
Harrison Chase
8f907161e3
Harrison/initial glossary (#61) 2022-11-04 08:02:21 -07:00
Harrison Chase
76aff023d7
FAISS and embedding support (#48)
also adds embeddings and an in memory docstore
2022-11-01 21:29:39 -07:00
Harrison Chase
5621ca7b07
Harrison/more documentation (#19) 2022-10-24 20:24:15 -07:00
Harrison Chase
18aeb72012 initial commit 2022-10-24 14:51:15 -07:00