Commit Graph

200 Commits

Author SHA1 Message Date
Harrison Chase
3e1901e1aa
gutenberg books (#946)
Co-authored-by: zanderchase <zander@unfold.ag>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
2023-02-08 12:00:47 -08:00
Ikko Eltociear Ashimine
6023d5be09
Update huggingface_hub.ipynb (#944)
HuggingFace -> Hugging Face
2023-02-08 11:05:28 -08:00
Harrison Chase
44ecec3896
Harrison/add roam loader (#939) 2023-02-08 00:35:33 -08:00
Ankush Gola
bc7e56e8df
Add asyncio support for LLM (OpenAI), Chain (LLMChain, LLMMathChain), and Agent (#841)
Supporting asyncio in langchain primitives allows for users to run them
concurrently and creates more seamless integration with
asyncio-supported frameworks (FastAPI, etc.)

Summary of changes:

**LLM**
* Add `agenerate` and `_agenerate`
* Implement in OpenAI by leveraging `client.Completions.acreate`

**Chain**
* Add `arun`, `acall`, `_acall`
* Implement them in `LLMChain` and `LLMMathChain` for now

**Agent**
* Refactor and leverage async chain and llm methods
* Add ability for `Tools` to contain async coroutine
* Implement async SerpaPI `arun`

Create demo notebook.

Open questions:
* Should all the async stuff go in separate classes? I've seen both
patterns (keeping the same class and having async and sync methods vs.
having class separation)
2023-02-07 21:21:57 -08:00
Vincent Elster
afc7f1b892
Fix typos (#929)
accomplisehd -> accomplished
2023-02-07 14:39:45 -08:00
Harrison Chase
637c0d6508
Harrison/obsidian (#920) 2023-02-06 22:21:16 -08:00
Harrison Chase
1e56879d38
Harrison/save faiss (#916)
Co-authored-by: Shrey Joshi <shreyjoshi2004@gmail.com>
2023-02-06 21:44:50 -08:00
Ankush Gola
6bd1529cb7
add GoogleDriveLoader (#914)
only deal with docs files for now
2023-02-06 21:44:35 -08:00
Harrison Chase
cea380174f
fix docs custom prompt template (#917) 2023-02-06 20:29:48 -08:00
Harrison Chase
87fad8fc00
analyze document (#731)
add analyze document chain, which does text splitting and then analysis
2023-02-06 20:02:19 -08:00
Harrison Chase
e2b834e427
Harrison/prompt template prefix (#888)
Co-authored-by: Gabriel Simmons <simmons.gabe@gmail.com>
2023-02-06 19:09:28 -08:00
Harrison Chase
f95cedc443
Harrison/sql rows (#915)
Co-authored-by: Jon Luo <20971593+jzluo@users.noreply.github.com>
2023-02-06 18:56:18 -08:00
Harrison Chase
2ec25ddd4c
add unstructured examples (#913) 2023-02-06 18:13:46 -08:00
Harrison Chase
71e662e88d
update docs (#905) 2023-02-06 00:26:20 -08:00
Harrison Chase
53d56d7650
Harrison/unstructured support (#903) 2023-02-05 23:02:07 -08:00
Harrison Chase
2a68be3e8d
chat vector db chain (#902) 2023-02-05 21:38:47 -08:00
James Briggs
8217a2f26c
Update pinecone init details in docs (#898)
PR to fix outdated environment details in the docs, see issue #897 

I added code comments as pointers to where users go to get API keys, and
where they can find the relevant environment variable.
2023-02-05 15:21:56 -08:00
Harrison Chase
a2b699dcd2
prompt template from string (#884) 2023-02-04 17:04:58 -08:00
Harrison Chase
0b9f086d36
Harrison/docs splitter (#879) 2023-02-03 15:09:13 -08:00
Zach Schillaci
4c79100b15
Correct prompt typo + update example for SQLDatabaseChain (#868)
See https://github.com/hwchase17/langchain/issues/821
2023-02-03 08:34:41 -08:00
Harrison Chase
3f48eed5bd
Harrison/milvus (#856)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
Signed-off-by: Frank Liu <frank.liu@zilliz.com>
Co-authored-by: Filip Haltmayer <81822489+filip-halt@users.noreply.github.com>
Co-authored-by: Frank Liu <frank@frankzliu.com>
2023-02-02 22:05:47 -08:00
Harrison Chase
fc0cfd7d1f
docs (#848) 2023-02-02 11:35:36 -08:00
Harrison Chase
23d5f64bda
Harrison/ngram example (#846)
Co-authored-by: Sean Spriggens <ssprigge@syr.edu>
2023-02-02 09:44:42 -08:00
Harrison Chase
0de55048b7
return code for pal (#844) 2023-02-02 08:47:20 -08:00
Harrison Chase
d564308e0f
rfc: instruct embeddings (#811)
Co-authored-by: seanaedmiston <seane999@gmail.com>
2023-02-02 08:44:02 -08:00
Istora Mandiri
06438794e1
Fix typo in textsplitter docs (#825) 2023-02-01 23:32:35 -08:00
Harrison Chase
7b4882a2f4
Harrison/tf embeddings (#817)
Co-authored-by: Ryohei Kuroki <10434946+yakigac@users.noreply.github.com>
2023-01-31 00:00:08 -08:00
Harrison Chase
94ae126747
return sql intermediate steps (#792) 2023-01-30 15:10:48 -08:00
Roy Williams
6086292252
Centralize logic for loading from LangChainHub, add ability to pin dependencies (#805)
It's generally considered to be a good practice to pin dependencies to
prevent surprise breakages when a new version of a dependency is
released. This commit adds the ability to pin dependencies when loading
from LangChainHub.

Centralizing this logic and using urllib fixes an issue identified by
some windows users highlighted in this video -
https://youtu.be/aJ6IQUh8MLQ?t=537
2023-01-30 14:52:17 -08:00
Harrison Chase
1ad7973cc6
Harrison/tool decorator (#790)
Co-authored-by: Jason Liu <jxnl@users.noreply.github.com>
Co-authored-by: Jason Liu <jason@jxnl.coA>
2023-01-28 18:26:24 -08:00
Harrison Chase
248c297f1b
Sample row in table info for SQLDatabase (#769) (#782)
The agents usually benefit from understanding what the data looks like
to be able to filter effectively. Sending just one row in the table info
allows the agent to understand the data before querying and get better
results.

---------

Co-authored-by: Francisco Ingham <>

---------

Co-authored-by: Francisco Ingham <fpingham@gmail.com>
2023-01-28 13:37:07 -08:00
Harrison Chase
c658f0aed3
Harrison/add to search (#778)
Co-authored-by: Enrico Shippole <enricoship@gmail.com>
2023-01-28 08:06:00 -08:00
Harrison Chase
a5d003f0c9
update notebook and make backwards compatible (#772) 2023-01-28 07:23:04 -08:00
Harrison Chase
b9ad214801
add docs for loading from hub (#763) 2023-01-27 07:10:26 -08:00
Harrison Chase
1b89a438cf
(wip) Harrison/serialize agents (#725) 2023-01-26 19:48:47 -08:00
Roy Williams
d2f882158f
Add type information for crawler.py (#738)
Added type information to `crawler.py` to make it safer to use and
understand.
2023-01-26 19:37:31 -08:00
Harrison Chase
bd0bf4e0a9
Harrison/generate blog post (#732)
Co-authored-by: Ren <yirenlu92@users.noreply.github.com>
2023-01-24 22:54:12 -08:00
scadEfUr
e3df8ab6dc
move hyde into chains (#728)
Co-authored-by: scadEfUr <>
2023-01-24 22:23:32 -08:00
Harrison Chase
0ffeabd14f
Harrison/serialize llm chain (#671) 2023-01-24 21:36:19 -08:00
Sam Hogan
499e54edda
fix typos in readme and text splitter docs (#720)
Fix typos in readme and TextSplitter documentation.
2023-01-24 10:59:23 -08:00
Николай Шангин
18b1466893
Fix not imported 'validator' (#715)
otherwise `@validator("input_variables")` do not work
2023-01-24 07:06:50 -08:00
Nicolas
66fd57878a
docs: Update vector_db_qa_with_sources.ipynb (#706) 2023-01-23 23:06:54 -08:00
Harrison Chase
fc4ad2db0f
langchain hub docs (#704)
Co-authored-by: scadEfUr <123224380+scadEfUr@users.noreply.github.com>
2023-01-23 23:06:23 -08:00
Harrison Chase
3a30e6daa8
Harrison/openai callback (#684) 2023-01-22 23:37:01 -08:00
Amos Ng
8baf6fb920
Update examples to fix execution problems (#685)
On the [Getting Started
page](https://langchain.readthedocs.io/en/latest/modules/prompts/getting_started.html)
for prompt templates, I believe the very last example

```python
print(dynamic_prompt.format(adjective=long_string))
```

should actually be

```python
print(dynamic_prompt.format(input=long_string))
```

The existing example produces `KeyError: 'input'` as expected

***

On the [Create a custom prompt
template](https://langchain.readthedocs.io/en/latest/modules/prompts/examples/custom_prompt_template.html#id1)
page, I believe the line

```python
Function Name: {kwargs["function_name"]}
```

should actually be

```python
Function Name: {kwargs["function_name"].__name__}
```

The existing example produces the prompt:

```
        Given the function name and source code, generate an English language explanation of the function.
        Function Name: <function get_source_code at 0x7f907bc0e0e0>
        Source Code:
        def get_source_code(function_name):
    # Get the source code of the function
    return inspect.getsource(function_name)

        Explanation:
```

***

On the [Example
Selectors](https://langchain.readthedocs.io/en/latest/modules/prompts/examples/example_selectors.html)
page, the first example does not define `example_prompt`, which is also
subtly different from previous example prompts used. For user
convenience, I suggest including

```python
example_prompt = PromptTemplate(
    input_variables=["input", "output"],
    template="Input: {input}\nOutput: {output}",
)
```

in the code to be copy-pasted
2023-01-22 14:49:25 -08:00
Samantha Whitmore
77e3d58922
ConversationEntityMemory: Chain which uses an entity extraction & sum… (#678)
…marization prompt to maintain a key-value store of memory information

cc @devennavani

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-01-22 10:10:02 -08:00
Ikko Eltociear Ashimine
64580259d0
Fix typo in hyde.ipynb (#688)
therefor -> therefore
2023-01-22 08:21:31 -08:00
Harrison Chase
e45f7e40e8
Harrison/few shot yaml (#682)
Co-authored-by: vintro <77507980+vintrocode@users.noreply.github.com>
2023-01-21 16:08:03 -08:00
Will Olson
2f57d18b25
Update hyperlink in Custom Prompt Template page (#677)
The current link points to a non-existent page. I've updated the link to
match what is on the "Create a custom example selector" page.

<img width="584" alt="Screen Shot 2023-01-21 at 10 33 05 AM"
src="https://user-images.githubusercontent.com/6773706/213879535-d8f2953d-ac37-448d-9b32-fdeb7b73cc32.png">
2023-01-21 16:03:21 -08:00
Harrison Chase
3d41af0aba
Harrison/load tools kwargs (#681)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
2023-01-21 16:03:02 -08:00
Harrison Chase
0b204d8c21
Harrison/quadrant (#665)
Co-authored-by: Kacper Łukawski <kacperlukawski@users.noreply.github.com>
2023-01-20 09:45:01 -08:00
Harrison Chase
d0fdc6da11
Harrison/bing wrapper (#656)
Co-authored-by: Enrico Shippole <henryshippole@gmail.com>
2023-01-19 14:48:30 -08:00
Charles Frye
bfb23f4608
typo bugfixes in getting started with prompts (#651)
tl;dr: input -> word, output -> antonym, rename to dynamic_prompt
consistently

The provided code in this example doesn't run, because the keys are
`word` and `antonym`, rather than `input` and `output`.

Also, the `ExampleSelector`-based prompt is named `few_shot_prompt` when
defined and `dynamic_prompt` in the follow-up example. The former name
is less descriptive and collides with an earlier example, so I opted for
the latter.

Thanks for making a really cool library!
2023-01-19 07:05:20 -08:00
John
3adc5227cd
typo (#650) 2023-01-19 07:03:11 -08:00
Harrison Chase
30abfc41c2
add instructions for saving loading (#642) 2023-01-18 00:19:05 -08:00
Harrison Chase
95720adff5
Add documentation for custom prompts for Agents (#631) (#640)
- Added a comment interpreting regex for `ZeroShotAgent`
- Added a note to the `Custom Agent` notebook

Co-authored-by: Sam Ching <samuel@duolingo.com>
2023-01-17 22:47:15 -08:00
Harrison Chase
6be5f4e4c4
Harrison/sql db chain (#641)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
2023-01-17 22:32:28 -08:00
Chetanya Rastogi
b550f57912
Fix the env variable for OpenAI Base Url (#639)
For using Azure OpenAI API, we need to set multiple env vars. But as can
be seen in openai package
[here](48b69293a3/openai/__init__.py (L35)),
the env var for setting base url is named `OPENAI_API_BASE` and not
`OPENAI_API_BASE_URL`. This PR fixes that part in the documentation.
2023-01-17 22:30:29 -08:00
Francis
b374d481c8
fix typo (#636)
there is a small typo in one of the docs.
2023-01-17 22:17:50 -08:00
Harrison Chase
3d43906572
Harrison/new api chain (#623)
Co-authored-by: Francisco Ingham <fpingham@gmail.com>
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>
2023-01-15 18:34:43 -08:00
Harrison Chase
1c71fadfdc
more complex sql chain (#619)
add a more complex sql chain that first subsets the necessary tables
2023-01-15 17:07:21 -08:00
Harrison Chase
2a54e73fec
bump version to 0063 (#616) 2023-01-14 08:09:25 -08:00
Francisco Ingham
1787c473b8
Custom prompt option for llm_bash and api chains (#612)
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>
2023-01-14 07:22:52 -08:00
Harrison Chase
9f9afbb6a8
add custom prompt for LLMMathChain and SQLDatabase chain (#605) 2023-01-13 06:28:51 -08:00
Sasmitha Manathunga
3e55f1474e
docs: fix typo (#604) 2023-01-12 21:36:03 -08:00
Rukmal Weerawarana
0f544a8811
Fix minor error in LLM documentation (#602) 2023-01-12 18:16:32 -08:00
Ikko Eltociear Ashimine
60dfe58325
Fix typo in vector_db_qa.ipynb (#597)
paramter -> parameter
2023-01-12 08:23:24 -08:00
Harrison Chase
950a81399a
bump version to 61 (#596) 2023-01-12 07:20:16 -08:00
Harrison Chase
d574bf0a27
add documentation on how to load different chain types (#595) 2023-01-12 06:47:38 -08:00
Harrison Chase
8ab09c18a1
Return source documents option in VectorDBQA (#585) (#592)
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>

Co-authored-by: Francisco Ingham <fpingham@gmail.com>
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>
2023-01-12 06:09:32 -08:00
Harrison Chase
7b6e7f6e12
bump to version 60 (#583) 2023-01-11 07:09:30 -08:00
Harrison Chase
f74ce7a104
Harrison/combine memories (#582)
Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
Co-authored-by: Diwank Singh Tomer <diwank.singh@gmail.com>
2023-01-11 06:08:58 -08:00
Harrison Chase
ffc7e04d44
Harrison/wolfram alpha (#579)
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
2023-01-11 05:52:19 -08:00
Harrison Chase
7de5139750
add example selector docs (#564) 2023-01-09 19:17:29 -08:00
Harrison Chase
b06a2a6191
improve documentation on how to pass in custom prompts (#561) 2023-01-08 19:20:13 -08:00
Harrison Chase
1192cc0767
smart text splitter (#530)
smart text splitter that iteratively tries different separators until it
works!
2023-01-08 15:11:10 -08:00
Harrison Chase
8dfad874a2
map rerank chain (#516)
add a chain that applies a prompt to all inputs and then returns not
only an answer but scores it

add examples for question answering and question answering with sources
2023-01-08 06:49:22 -08:00
Nicolas
948eee9fe1
Docs: side menu to match the order (llms) (#557)
Small quick fix:

Suggest making the order of the menu the same as it is written on the
page (Getting Started -> Key Concepts). Before the menu order was not
the same as it was on the page. Not sure if this is the only place the
menu is affected.

Mismatch is found here:
https://langchain.readthedocs.io/en/latest/modules/llms.html
2023-01-06 09:34:08 -08:00
Harrison Chase
823a44ef80
bump to 0058 (#556) 2023-01-06 07:58:38 -08:00
Harrison Chase
74932f2516
RFC: conversational agent (#464)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
2023-01-06 07:25:55 -08:00
Harrison Chase
e64ed7b975
Harrison/tools priority (#554)
Co-authored-by: Yong723 <50616781+Yongtae723@users.noreply.github.com>
2023-01-06 06:56:11 -08:00
Harrison Chase
4974f49bb7
add return_direct flag to tool (#537)
adds a return_direct flag to tools, which just returns the tool output
as the final output
2023-01-06 06:40:32 -08:00
Harrison Chase
9753bccc71
Feature: linkcheck-action (#534) (#542)
- Add support for local build and linkchecking of docs
- Add GitHub Action to automatically check links before prior to
publication
- Minor reformat of Contributing readme
- Fix existing broken links

Co-authored-by: Hunter Gerlach <hunter@huntergerlach.com>

Co-authored-by: Hunter Gerlach <HunterGerlach@users.noreply.github.com>
Co-authored-by: Hunter Gerlach <hunter@huntergerlach.com>
2023-01-04 21:39:50 -08:00
Harrison Chase
73f7ebd9d1
Harrison/sqlalchemy cache store (#536)
Co-authored-by: Jason Gill <jasongill@gmail.com>
2023-01-04 18:38:15 -08:00
Rubens Mau
020e73017b
Updated embeddings.ipynb (#531)
updated embeddings.ipynb
2023-01-04 10:43:52 -08:00
Ikko Eltociear Ashimine
ca9aaac36e
Fix typo in key_concepts.md (#535)
therefor -> therefore
2023-01-04 10:43:02 -08:00
Harrison Chase
9e04c34e20
Add BaseCallbackHandler and CallbackManager (#478)
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
2023-01-04 07:54:25 -08:00
Harrison Chase
0db05b6725
Harrison/add human prefix (#520)
Co-authored-by: Andrew Huang <jhuang16888@gmail.com>
2023-01-03 08:03:50 -08:00
Harrison Chase
03f185bcd5
more robust handling for max iterations (#514)
add a `generate` method which makes one final forward pass through the
llm
2023-01-03 07:46:08 -08:00
Harrison Chase
40326c698c
unify argument name (#513)
unify names in map reduce and refine chains to just be
return_intermediate_steps

also unify the return key
2023-01-03 07:45:08 -08:00
Harrison Chase
985496f4be
Docs refactor (#480)
Big docs refactor! Motivation is to make it easier for people to find
resources they are looking for. To accomplish this, there are now three
main sections:

- Getting Started: steps for getting started, walking through most core
functionality
- Modules: these are different modules of functionality that langchain
provides. Each part here has a "getting started", "how to", "key
concepts" and "reference" section (except in a few select cases where it
didnt easily fit).
- Use Cases: this is to separate use cases (like summarization, question
answering, evaluation, etc) from the modules, and provide a different
entry point to the code base.

There is also a full reference section, as well as extra resources
(glossary, gallery, etc)

Co-authored-by: Shreya Rajpal <ShreyaR@users.noreply.github.com>
2023-01-02 08:24:09 -08:00
Harrison Chase
c104d507bf
Harrison/improve data augmented generation docs (#390)
Co-authored-by: cameronccohen <cameron.c.cohen@gmail.com>
Co-authored-by: Cameron Cohen <cameron.cohen@quantco.com>
2022-12-20 22:24:08 -05:00
Harrison Chase
b5d8434a50
Harrison/improve chain docs (#251) 2022-12-03 13:28:50 -08:00
Harrison Chase
d3a7429f61
(WIP) agents (#171) 2022-11-22 06:16:26 -08:00
Harrison Chase
c02eb199b6
add few shot example (#148) 2022-11-19 20:32:45 -08:00
Harrison Chase
b1b6b27c5f
Harrison/redo docs (#130)
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2022-11-13 20:13:23 -08:00
Harrison Chase
5e76c12455
Harrison/fix docs (#115) 2022-11-10 08:59:51 -08:00
Harrison Chase
76aff023d7
FAISS and embedding support (#48)
also adds embeddings and an in memory docstore
2022-11-01 21:29:39 -07:00
Harrison Chase
5621ca7b07
Harrison/more documentation (#19) 2022-10-24 20:24:15 -07:00
Harrison Chase
18aeb72012 initial commit 2022-10-24 14:51:15 -07:00