Commit Graph

483 Commits (main)

Author SHA1 Message Date
Harrison Chase 7f76a1189c
bump version to 0.0.70 (#744) 1 year ago
Harrison Chase 2ba1128095
Harrison/backwards compat (#740) 1 year ago
Francisco Ingham f9ddcb5705
Hotfix: distance_func and collection_name must not be in kwargs (#735)
If `distance_func` and `collection_name` are in `kwargs` they are sent
to the `QdrantClient` which results in an error being raised.

Co-authored-by: Francisco Ingham <>
1 year ago
Amos Ng fa6826e417
Fix sqlalchemy warnings when running tests (#733)
This has been bugging me when running my own tests that call langchain
methods :P
1 year ago
Harrison Chase 9194a8be89
add stop to stream (#729) 1 year ago
scadEfUr e3df8ab6dc
move hyde into chains (#728)
Co-authored-by: scadEfUr <>
1 year ago
Harrison Chase 0ffeabd14f
Harrison/serialize llm chain (#671) 1 year ago
Feynman Liang 2824f36401
Add namespace to Pinecone.from_index (#716)
Resolves https://github.com/hwchase17/langchain/issues/718
1 year ago
Kacper Łukawski d4f719c34b
Convert numpy arrays to lists in HuggingFaceEmbeddings (#714)
`SentenceTransformer` returns a NumPy array, not a `List[List[float]]`
or `List[float]` as specified in the interface of `Embeddings`. That PR
makes it consistent with the interface.
1 year ago
Kacper Łukawski 97c3544a1e
Hotfix: Qdrant.from_text embeddings (#713)
I'm providing a hotfix for Qdrant integration. Calculating a single
embedding to obtain the vector size was great idea. However, that change
introduced a bug trying to put only that single embedding into the
database. It's fixed. Right now all the embeddings will be pushed to
Qdrant.
1 year ago
Feynman Liang 3a38604f07
Fix typo (#705) 1 year ago
Harrison Chase fc4ad2db0f
langchain hub docs (#704)
Co-authored-by: scadEfUr <123224380+scadEfUr@users.noreply.github.com>
1 year ago
Scott Leibrand 34932dd211
remove legacy embedding model name (#703)
Now that OpenAI has deprecated all embeddings models except
text-embedding-ada-002, we should stop specifying a legacy embedding
model in the example. This will also avoid confusion from people (like
me) trying to specify model="text-embedding-ada-002" and having that
erroneously expanded to text-search-text-embedding-ada-002-query-001
1 year ago
scadEfUr 4aba0abeaa
added common prompt load method (#699)
Co-authored-by: scadEfUr
1 year ago
xloem 36b6b3cdf6
HuggingFacePipeline: Forward model_kwargs. (#696)
Since the tokenizer and model are constructed manually, model_kwargs
needs to
be passed to their constructors. Additionally, the pipeline has a
specific
named parameter to pass these with, which can provide forward
compatibility if
they are used for something other than tokenizer or model construction.
1 year ago
Harrison Chase 3a30e6daa8
Harrison/openai callback (#684) 1 year ago
Harrison Chase aef82f5d59
fix whitespace for conversational agent (#690) 1 year ago
Harrison Chase 86dbdb118b
Harrison/serpapi extra tools (#691)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
1 year ago
Harrison Chase cbc146720b
verbose flag (#683) 1 year ago
Harrison Chase 27cef0870d
bump version to 0.0.67 (#689) 1 year ago
Samantha Whitmore 77e3d58922
ConversationEntityMemory: Chain which uses an entity extraction & sum… (#678)
…marization prompt to maintain a key-value store of memory information

cc @devennavani

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
1 year ago
dham e04b063ff4
add faiss local saving/loading (#676)
- This uses the faiss built-in `write_index` and `load_index` to save
and load faiss indexes locally
- Also fixes #674
- The save/load functions also use the faiss library, so I refactored
the dependency into a function
1 year ago
Harrison Chase e45f7e40e8
Harrison/few shot yaml (#682)
Co-authored-by: vintro <77507980+vintrocode@users.noreply.github.com>
1 year ago
Harrison Chase a2eeaf3d43
strip whitespace (#680) 1 year ago
Harrison Chase 3d41af0aba
Harrison/load tools kwargs (#681)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
1 year ago
Harrison Chase 0b204d8c21
Harrison/quadrant (#665)
Co-authored-by: Kacper Łukawski <kacperlukawski@users.noreply.github.com>
1 year ago
Harrison Chase 983b73f47c
add search kwargs (#664) 1 year ago
vertinski 65f3a341b0
Prompt fix for empty intermediate steps in summarization (#660)
Adding quotation marks around {text} avoids generating empty or
completely random responses from OpenAI davinci-003. Empty or completely
unrelated intermediate responses in summarization messes up the final
result or makes it very inaccurate.
The error from OpenAI would be: "The model predicted a completion that
begins with a stop sequence, resulting in no output. Consider adjusting
your prompt or stop sequences."
This fix corrects the prompting for summarization chain. This works on
API too, the images are for demonstrative purposes.
This approach can be applied to other similar prompts too. 

Examples:

1) Without quotation marks
![Screenshot from 2023-01-20
07-18-19](https://user-images.githubusercontent.com/22897470/213624365-9dfc18f9-5f3f-45d2-abe1-56de67397e22.png)

2) With quotation marks
![Screenshot from 2023-01-20
07-18-35](https://user-images.githubusercontent.com/22897470/213624478-c958e742-a4a7-46fe-a163-eca6326d9dae.png)
1 year ago
iocuydi 69998b5fad
Add ids parameter for pinecone from_texts / add_texts (#659)
Allow optionally specifying a list of ids for pinecone rather than
having them randomly generated.
This also permits editing the embedding/metadata of existing pinecone
entries, by id.
1 year ago
Harrison Chase 54d7f1c933
fix caching (#658) 1 year ago
Harrison Chase d0fdc6da11
Harrison/bing wrapper (#656)
Co-authored-by: Enrico Shippole <henryshippole@gmail.com>
1 year ago
iocuydi 207e319a70
Add search_kwargs option for VectorDBQAWithSourcesChain (#657)
Allows for passing additional vectorstore params like namespace, etc. to
VectorDBQAWithSourcesChain

Example:
`chain = VectorDBQAWithSourcesChain.from_llm(OpenAI(temperature=0),
vectorstore=store, search_kwargs={"namespace": namespace})`
1 year ago
Harrison Chase 052c361031
pinecone docstring (#654) 1 year ago
Harrison Chase 95720adff5
Add documentation for custom prompts for Agents (#631) (#640)
- Added a comment interpreting regex for `ZeroShotAgent`
- Added a note to the `Custom Agent` notebook

Co-authored-by: Sam Ching <samuel@duolingo.com>
1 year ago
Harrison Chase 6be5f4e4c4
Harrison/sql db chain (#641)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
1 year ago
Harrison Chase 4d4cff0530
Harrison/cohere experimental (#638)
Co-authored-by: inyourhead <44607279+xettrisomeman@users.noreply.github.com>
1 year ago
Sasmitha Manathunga 5c97f70bf1
Fix CohereError: embed is not an available endpoint on this model (#637)
Running the Cohere embeddings example from the docs:

```python
from langchain.embeddings import CohereEmbeddings
embeddings = CohereEmbeddings(cohere_api_key= cohere_api_key)

text = "This is a test document."
query_result = embeddings.embed_query(text)
doc_result = embeddings.embed_documents([text])
```

I get the error:

```bash
CohereError(message=res['message'], http_status=response.status_code, headers=response.headers)      
cohere.error.CohereError: embed is not an available endpoint on this model
```

This is because the `model` string is set to `medium` which is not
currently available.

From the Cohere docs:

> Currently available models are small and large (default)
1 year ago
Francisco Ingham b929fd9f59
Exclude reference to 'example' in api prompt (#629)
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>
1 year ago
Harrison Chase 3d43906572
Harrison/new api chain (#623)
Co-authored-by: Francisco Ingham <fpingham@gmail.com>
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>
1 year ago
Harrison Chase 1c71fadfdc
more complex sql chain (#619)
add a more complex sql chain that first subsets the necessary tables
1 year ago
Harrison Chase 49b3d6c78c
Harrison/wiki update (#622)
Co-authored-by: Rubens Mau <rubensmau@gmail.com>
1 year ago
Harrison Chase 1ac3319e45
simplify parsing of the final answer (#621) 1 year ago
Harrison Chase 2a54e73fec
bump version to 0063 (#616) 1 year ago
Nicolas 91d7fd20ae
feat: add custom prompt for QAEvalChain chain (#610)
I originally had only modified the `from_llm` to include the prompt but
I realized that if the prompt keys used on the custom prompt didn't
match the default prompt, it wouldn't work because of how `apply` works.

So I made some changes to the evaluate method to check if the prompt is
the default and if not, it will check if the input keys are the same as
the prompt key and update the inputs appropriately.

Let me know if there is a better way to do this.

Also added the custom prompt to the QA eval notebook.
1 year ago
Francisco Ingham 1787c473b8
Custom prompt option for llm_bash and api chains (#612)
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>
1 year ago
Harrison Chase 67808bad0e
expose more serpapi parameters (#609) 1 year ago
Harrison Chase 9f9afbb6a8
add custom prompt for LLMMathChain and SQLDatabase chain (#605) 1 year ago
Smit Shah a87a2aacaa
[Minor Fix] Fix spacy TextSplitter init (#606) 1 year ago
babbldev b5eb91536a
Added filter argument to pinecone queries, fixes #600 (#601)
Added filter argument to similarity_search() and
similarity_search_with_score()

Co-authored-by: Sam Cartford (MBP) <cartford@hey.com>
1 year ago
Harrison Chase d574bf0a27
add documentation on how to load different chain types (#595) 1 year ago
Harrison Chase 8ab09c18a1
Return source documents option in VectorDBQA (#585) (#592)
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>

Co-authored-by: Francisco Ingham <fpingham@gmail.com>
Co-authored-by: lesscomfortable <pancho_ingham@hotmail.com>
1 year ago
Harrison Chase 4c6c5f0391
wolfram alpha improvements (#591)
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
1 year ago
Harrison Chase a5ee7de650
pinecone changes (#590)
Co-authored-by: Smit Shah <who828@gmail.com>
Co-authored-by: iocuydi <46613640+iocuydi@users.noreply.github.com>
1 year ago
Harrison Chase 3f2ea5c35e
Harrison/load from hub (#580) 1 year ago
Harrison Chase f74ce7a104
Harrison/combine memories (#582)
Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
Co-authored-by: Diwank Singh Tomer <diwank.singh@gmail.com>
1 year ago
Harrison Chase 2aa08631cb
add similarity score method to faiss (#574)
adds `similarity_search_with_score` to faiss wrapper
1 year ago
Harrison Chase 5ba46f6d0c
Harrison/namespace pinecone (#581)
Co-authored-by: mmorzywolek <89693033+mmorzywolek@users.noreply.github.com>
1 year ago
Harrison Chase ffc7e04d44
Harrison/wolfram alpha (#579)
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
1 year ago
Yong723 94c06c55e8
modify docstring (#569)
Sorry for the detail. this is a correction to the docstring.
1 year ago
Yong723 e1f3871a78
fix typo (#570)
I found a typo, which might be important for a conversational Agent.

if My PR is wrong, I am so sorry
1 year ago
Harrison Chase 1511606799
Harrison/fix splitting (#563)
fix issue where text splitting could possibly create empty docs
1 year ago
Harrison Chase 1192cc0767
smart text splitter (#530)
smart text splitter that iteratively tries different separators until it
works!
1 year ago
Harrison Chase 8dfad874a2
map rerank chain (#516)
add a chain that applies a prompt to all inputs and then returns not
only an answer but scores it

add examples for question answering and question answering with sources
1 year ago
Harrison Chase 823a44ef80
bump to 0058 (#556) 1 year ago
Benjamin 42d5d988fa
add openai logit bias (#553)
Add
[`logit_bias`](https://beta.openai.com/docs/api-reference/completions/create#completions/create-logit_bias)
params to OpenAI

See [here](https://beta.openai.com/tokenizer) for the tokenizer.

NB: I see that others (like Cohere) have the same parameter, but since I
don't have an access to it, I don't want to make a mistake.

---

Just to make sure the default "{}" works for openai:
```
from langchain.llms import OpenAI

OPENAI_API_KEY="XXX"

llm = OpenAI(openai_api_key=OPENAI_API_KEY)
llm.generate('Write "test":')

llm = OpenAI(openai_api_key=OPENAI_API_KEY, logit_bias={'9288': -100, '1332': -100, '14402': -100, '6208': -100})
llm.generate('Write "test":')
```
1 year ago
Harrison Chase 9833fcfe32
fix caching (#555) 1 year ago
Harrison Chase 74932f2516
RFC: conversational agent (#464)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
1 year ago
Harrison Chase 330a5b42d4
fix map reduce chain (#550) 1 year ago
Diwank Singh Tomer ba0cbb4a41
Add finish reason to Generation for usage downstream (#526)
Add `finish_reason` to `Generation` as well as extend
`BaseOpenAI._generate` to include it in the output. This can be useful
for usage in downstream tasks when we need to filter for only
generations that finished because of `"stop"` for example. Maybe we
should add this to `LLMChain` as well?

For more details, see
https://beta.openai.com/docs/guides/completion/best-practices

Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
1 year ago
Harrison Chase 4974f49bb7
add return_direct flag to tool (#537)
adds a return_direct flag to tools, which just returns the tool output
as the final output
1 year ago
Harrison Chase 0c2f7d8da1
changes to qa chain (#543) 1 year ago
Harrison Chase 5aefc2b7ce
add handling on error (#541) 1 year ago
Harrison Chase 1631981f84
Harrison/fix and test caching (#538) 1 year ago
Harrison Chase 73f7ebd9d1
Harrison/sqlalchemy cache store (#536)
Co-authored-by: Jason Gill <jasongill@gmail.com>
1 year ago
Yongtae723 f48ab642be
replace forbid into ignore (#539)
this is the second PR of #519.
in #519 I suggested deleting Extra.forbid.
I was very confused but I replaced Extra.forbid to Extra.ignore, which
is the default of pydantic.


Since the
[BaseLLM](4b7b8229de/langchain/llms/base.py (L20))
from which it is inherited is set in Extra.forbid, I wanted to avoid
having the Extra.forbid settings inherited by simply deleting it.
1 year ago
Yongtae723 4b7b8229de
add logger (#529)
As talking #519, I made 2 PRs.

this is the first PR for adding a logger.

I am concerned about the following two points and would appreciate your
opinion.

1. Since the logger is not formatted, the statement itself is output
like a print statement, and I thought it was difficult to understand
that it was a warning, so I put WARNING! at the beginning of the warning
statement. After the logger formatting is done properly, the word
WARNING can be repeated.
2. Statement `Please confirm that {field_name} is what you intended.`
can be replaced like `If {field_name} is intended parameters, enter it
to model_kwargs`
thank you!

Yongtae
1 year ago
Harrison Chase 9e04c34e20
Add BaseCallbackHandler and CallbackManager (#478)
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
1 year ago
Harrison Chase 0db05b6725
Harrison/add human prefix (#520)
Co-authored-by: Andrew Huang <jhuang16888@gmail.com>
1 year ago
Harrison Chase 03f185bcd5
more robust handling for max iterations (#514)
add a `generate` method which makes one final forward pass through the
llm
1 year ago
Harrison Chase 40326c698c
unify argument name (#513)
unify names in map reduce and refine chains to just be
return_intermediate_steps

also unify the return key
1 year ago
Harrison Chase 985496f4be
Docs refactor (#480)
Big docs refactor! Motivation is to make it easier for people to find
resources they are looking for. To accomplish this, there are now three
main sections:

- Getting Started: steps for getting started, walking through most core
functionality
- Modules: these are different modules of functionality that langchain
provides. Each part here has a "getting started", "how to", "key
concepts" and "reference" section (except in a few select cases where it
didnt easily fit).
- Use Cases: this is to separate use cases (like summarization, question
answering, evaluation, etc) from the modules, and provide a different
entry point to the code base.

There is also a full reference section, as well as extra resources
(glossary, gallery, etc)

Co-authored-by: Shreya Rajpal <ShreyaR@users.noreply.github.com>
1 year ago
Keiji Kanazawa c5f0af9398
Minor docstring update (#507)
Update `model=` to `model_name=`.

No need to credit me for this 😄
1 year ago
Harrison Chase 0072686aab
Harrison/new search engine (#477)
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
1 year ago
Harrison Chase 3e41ab7bff
check keys before using (#475) 1 year ago
Harrison Chase 0f1df0dc2c
bump to version 0.0.52 (#470) 1 year ago
Parth Chadha e88e66f982
Pass verbose argument to LLMChains when using *DocumentsChain (#458)
When using chains such as Summarization chain (`load_summarize_chain`),
the verbose flag wasn't propagated to the `LLMChain`.
1 year ago
Harrison Chase d0f194de73
add logic for agent stopping (#420) 1 year ago
Harrison Chase c65efd2986
fix llm math prompt (#466)
basically, it didnt realize that the question was over after the input
and would some times hallucinate more input
1 year ago
Harrison Chase 95157d0aad
Add schema property to sql database utility class (#448) (#462)
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>

Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
Co-authored-by: Nuno Campos <nuno@boringbits.io>
Co-authored-by: Diwank Singh Tomer <diwank.singh@gmail.com>
1 year ago
Harrison Chase 2b84e5cda3
Harrison/fix memory and serp (#457)
Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
1 year ago
Harrison Chase 55007e71be
add output key for memory (#443)
this allows chains that return multiple values to use memory
1 year ago
Harrison Chase 5208bb8c36
make tools editable (#445)
use dataclass instead of namedtuple, which makes it editable

add example in notebook
1 year ago
Harrison Chase 5cc6bf1a9c
fix regex parser (#446) 1 year ago
Ikko Ashimine f3c3288761
chore: fix typo in prompt.py (#447)
seperator -> separator
1 year ago
Harrison Chase 9ec01dfc16
regex output parser (#435) 1 year ago
Harrison Chase c994ce6b7f
Harrison/serp api imp (#444)
improve serp api

Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>
1 year ago
Harrison Chase ffe35c396c
unify return types across map-reduce and refine (#442) 1 year ago
Harrison Chase 0c5d3fd894
version 0.0.49 (#436) 1 year ago
Harrison Chase f8b605293f
Harrison/improve memory (#432)
add AI prefix

add new type of memory

Co-authored-by: Jason <chisanch@usc.edu>
1 year ago
Harrison Chase 150b67de10
Harrison/weaviate improvements (#433)
Co-authored-by: Connor Shorten <connorshorten300@gmail.com>
1 year ago
Harrison Chase b7566b5ec3
Harrison/return intermediate steps (#428) 1 year ago
Harrison Chase b50a56830d
Harrison/evaluation notebook (#426) 1 year ago
Harrison Chase 97f4000d3a
fix react docstore (#427) 1 year ago
Harrison Chase ee3b8e89b3
better parsing of agent output (#418) 1 year ago
Harrison Chase 0d7aa1ee99
Harrison/docs to index (#419)
Add method for going directly from documents to VectorStores

Update notebook to showcase this functionality
1 year ago
Harrison Chase 20959d8c36
check memory variables (#411)
can have multiple input keys, if some come from memory
1 year ago
altryne f990395211
Readme typos (#409)
I was honored by the twitter mention, so used PyCharm to try and... help
docs even a little bit.
Mostly typo-s and correct spellings. 

PyCharm really complains about "really good" being used all the time and
recommended alternative wordings haha
1 year ago
Shreya Rajpal f40b3ce347
Updated VectorDBQA docs to updated argument name (#405) 1 year ago
Samantha Whitmore 6bc8ae63ef
Add Redis cache implementation (#397)
I'm using a hash function for the key just to make sure its length
doesn't get out of hand, otherwise the implementation is quite similar.
1 year ago
mrbean 136f759492
Mrbean/support timeout (#398)
Add support for passing in a request timeout to the API
1 year ago
Harrison Chase 6b60c509ac
(WIP) add HyDE (#393)
Co-authored-by: cameronccohen <cameron.c.cohen@gmail.com>
Co-authored-by: Cameron Cohen <cameron.cohen@quantco.com>
1 year ago
Keiji Kanazawa 543db9c2df
Add Azure OpenAI LLM (#395)
Hi!  This PR adds support for the Azure OpenAI service to LangChain.

I've tried to follow the contributing guidelines.

Co-authored-by: Keiji Kanazawa <{ID}+{username}@users.noreply.github.com>
1 year ago
Harrison Chase c104d507bf
Harrison/improve data augmented generation docs (#390)
Co-authored-by: cameronccohen <cameron.c.cohen@gmail.com>
Co-authored-by: Cameron Cohen <cameron.cohen@quantco.com>
1 year ago
Harrison Chase 6be5747466
RFC: add cache override to LLM class (#379) 1 year ago
Harrison Chase 46c428234f
MMR example selector (#377)
implement max marginal relevance example selector
1 year ago
Harrison Chase ffed5e0056
Harrison/jinja formatter (#385)
Co-authored-by: Benjamin <BenderV@users.noreply.github.com>
1 year ago
mrbean fc66a32c6f
fix docstring (#383)
![Screenshot 2022-12-19 at 11 06 48
AM](https://user-images.githubusercontent.com/43734688/208468970-5cb9bafb-f535-486e-b41f-312a2f9ffffb.png)
1 year ago
Harrison Chase cf98f219f9
Harrison/tools exp (#372) 1 year ago
Harrison Chase e7b625fe03
fix text splitter (#375) 1 year ago
Harrison Chase 3474f39e21
Harrison/improve cache (#368)
make it so everything goes through generate, which removes the need for
two types of caches
1 year ago
Ankush Gola 8d0869c6d3
change run to use args and kwargs (#367)
Before, `run` was not able to be called with multiple arguments. This
expands the functionality.
1 year ago
Harrison Chase a7084ad6e4
Harrison/version 0040 (#366) 1 year ago
mrbean 50257fce59
Support Streaming Tokens from OpenAI (#364)
https://github.com/hwchase17/langchain/issues/363

@hwchase17 how much does this make you want to cry?
1 year ago
mrbean fe6695b9e7
Add HuggingFacePipeline LLM (#353)
https://github.com/hwchase17/langchain/issues/354

Add support for running your own HF pipeline locally. This would allow
you to get a lot more dynamic with what HF features and models you
support since you wouldn't be beholden to what is hosted in HF hub. You
could also do stuff with HF Optimum to quantize your models and stuff to
get pretty fast inference even running on a laptop.
1 year ago
Benjamin 85c1bd2cd0
add sqlalchemy generic cache (#361)
Created a generic SQLAlchemyCache class to plug any database supported
by SQAlchemy. (I am using Postgres).
I also based the class SQLiteCache class on this class SQLAlchemyCache.

As a side note, I'm questioning the need for two distinct class
LLMCache, FullLLMCache. Shouldn't we merge both ?
1 year ago
Harrison Chase 809a9f485f
Harrison/new version (#362) 1 year ago
Harrison Chase 750edfb440
add optional collapse prompt (#358) 1 year ago
Harrison Chase 2dd895d98c
add openai tokenizer (#355) 1 year ago
Harrison Chase c1b50b7b13
Harrison/map reduce merge (#344)
Co-authored-by: John Nay <JohnNay@users.noreply.github.com>
1 year ago
Harrison Chase ed143b598f
improve openai embeddings (#351)
add more formal support for explicitly specifying each model, but in a
backwards compatible way
1 year ago
Harrison Chase 78b31e5966
Harrison/cache (#343) 1 year ago
Harrison Chase 8cf62ce06e
Harrison/single input (#347)
allow passing of single input into chain

Co-authored-by: thepok <richterthepok@yahoo.de>
1 year ago
Harrison Chase e26b6f9c89
fix batching (#339) 1 year ago
Harrison Chase 996b5a3dfb
Harrison/llm final stuff (#332) 1 year ago
Harrison Chase 9bb7195085
Harrison/llm saving (#331)
Co-authored-by: Akash Samant <70665700+asamant21@users.noreply.github.com>
1 year ago
Harrison Chase 595cc1ae1a
RFC: more complete return (#313)
Co-authored-by: Andrew Williamson <awilliamson10@indstate.edu>
Co-authored-by: awilliamson10 <aw.williamson10@gmail.com>
1 year ago
Harrison Chase 8861770bd0
expose get_num_tokens method (#327) 1 year ago
thepok 137356dbec
-1 max token description for openai (#330) 1 year ago
Harrison Chase a7c8e37e77
Harrison/token counts (#311)
Co-authored-by: thepok <richterthepok@yahoo.de>
1 year ago
Shobith Alva 19a9fa16a9
Add `clear()` method for `Memory` (#305)
a simple helper to clear the buffer in `Conversation*Memory` classes
1 year ago
Harrison Chase e02d6b2288
beta: logger (#307) 1 year ago
Harrison Chase 36b4c58acf
expose more stuff (#306) 1 year ago
Harrison Chase 7827f0a844
fix typing (int -> float) (#308) 1 year ago
Harrison Chase 853894dd47
add moderation chain (#299) 1 year ago
andersenchen 5267ebce2d
Add LLMCheckerChain (#281)
Implementation of https://github.com/jagilley/fact-checker. Works pretty
well.

<img width="993" alt="Screenshot 2022-12-07 at 4 41 47 PM"
src="https://user-images.githubusercontent.com/101075607/206302751-356a19ff-d000-4798-9aee-9c38b7f532b9.png">

Verifying this manually:
1. "Only two kinds of egg-laying mammals are left on the planet
today—the duck-billed platypus and the echidna, or spiny anteater."
https://www.scientificamerican.com/article/extreme-monotremes/
2. "An [Echidna] egg weighs 1.5 to 2 grams (0.05 to 0.07
oz)[[19]](https://en.wikipedia.org/wiki/Echidna#cite_note-19) and is
about 1.4 centimetres (0.55 in) long."
https://en.wikipedia.org/wiki/Echidna#:~:text=sleep%20is%20suppressed.-,Reproduction,a%20reptile%2Dlike%20egg%20tooth.
3. "A [platypus] lays one to three (usually two) small, leathery eggs
(similar to those of reptiles), about 11 mm (7⁄16 in) in diameter and
slightly rounder than bird eggs."
https://en.wikipedia.org/wiki/Platypus#:~:text=It%20lays%20one%20to%20three,slightly%20rounder%20than%20bird%20eggs.
4. Therefore, an Echidna is the mammal that lays the biggest eggs.


cc @hwchase17
1 year ago
Samantha Whitmore b10be842f6
ChatGPT Clone: adding ConversationBufferWindowMemory to replicate vir… (#288)
…tual env example
1 year ago
Harrison Chase e2e501aa06
Harrison/version 0032 (#283) 1 year ago
Harrison Chase e9b1c8cdfa
Harrison/base combine doc chain (#264) 1 year ago
Harrison Chase 834b391792
update notebooks (#275) 1 year ago
Harrison Chase 3c1c7ba672
update branch name in gha (#274) 1 year ago
Akash Samant 48b093823e
Add a Transformation Chain (#257)
Arbitrary transformation chains that can be used to add dictionary
extractions from llms/other chains
1 year ago
coyotespike b7bef36ee1
BashChain (#260)
Love the project, a ton of fun!

I think the PR is pretty self-explanatory, happy to make any changes! I
am working on using it in an `LLMBashChain` and may update as that
progresses.

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
1 year ago
Harrison Chase 28be37f470
LLMRequestsChain (#267) 1 year ago
John McDonnell 68666d6a22
Gracefully degrade when model asks for nonexistent tool (#268)
Not yet tested, but very simple change, assumption is that we're cool
with just producing a generic output when tool is not found
1 year ago
Harrison Chase 2163d064f3
add return of ids (#254)
not actually sure the desired return in add_example to example selector
is actually general/good - whats the use case?
1 year ago
Harrison Chase 8cba5b791a
hotfix for api logging (#262) 1 year ago
Harrison Chase 5cd6956d58
Harrison/version 0028 (#259) 1 year ago
Harrison Chase f5c665a544
combine python files (#256) 1 year ago
Harrison Chase 988cb51a7c
fix out of date docs (#255) 1 year ago
Harrison Chase 9481a23314
stop using chained input except in agent (#249) 1 year ago
Harrison Chase ac2c2f6f28
Harrison/delete bad code (#253) 1 year ago
Harrison Chase db58032973
introduce output parser (#250) 1 year ago
Harrison Chase a9ce04201f
Harrison/improve usability of api chain (#247)
improve usability of api chain
1 year ago
Harrison Chase c897bd6cbd
api chain (#246)
Co-authored-by: Subhash Ramesh <33400216+thecooltechguy@users.noreply.github.com>
1 year ago
Harrison Chase 8145c79fd8
bump version to 0.0.27 (#244) 1 year ago
Harrison Chase 78a29f1060
text world agent (#240) 1 year ago
Xupeng (Tony) Tong bb4bf9d6d0
chore: minor clean up / formatting (#233)
to get familiarize with the project
1 year ago
Harrison Chase 473943643e
bump version 0026 (#235) 1 year ago
Harrison Chase 3ca2c8d6c5
allow passing of stop params into openai (#232) 1 year ago
Harrison Chase 347fc49d4d
Harrison/combine documents chain (#212)
combine documents chain powering vector db qa with sources chain
1 year ago
Harrison Chase ab9abf53b7
Harrison/version 0025 (#227) 1 year ago
Harrison Chase ca2394028f
move search to not be a chain (#226) 1 year ago
Harrison Chase b19a73be26
pal chain touch ups (#225)
expose PAL in main entrypoint
1 year ago
Andrew Gleave ea67c049f0
Support SQL statements that return no results (#222)
Adds support for statements such as insert, update etc which do not
return any rows.

`engine.execute` is deprecated and so execution has been updated to use
`connection.exec_driver_sql` as-per:


https://docs.sqlalchemy.org/en/14/core/connections.html#sqlalchemy.engine.Engine.execute
1 year ago
Akash Samant d368c43648
Bug Fix (#221)
Quick bug fix for semantic similarity vector injection
1 year ago
Harrison Chase 1db7b18341
bump version to 0.0.24 (#220) 1 year ago
Harrison Chase 1b9b8efbc9
pal chain (#207)
from https://arxiv.org/pdf/2211.10435.pdf
1 year ago
Shyamal H Anadkat de4b255c1f
Switch default openai model to text-davinci-003 (#215) 1 year ago
Harrison Chase 0568998166
Harrison/fix react stateful (#219)
fix issue with react being stateful
1 year ago
Harrison Chase 03c7140228
fix self ask template (#216) 1 year ago
Harrison Chase cf3569fb1b
remove check (#217)
doesnt do much
1 year ago
Hansen Qian a39c998342
Add chain name to verbose logging (#214)
Adds some context over what chain is running, thereby making it more
obvious how different chains are entered and existed

<img width="867" alt="Screen Shot 2022-11-28 at 11 55 34 AM"
src="https://user-images.githubusercontent.com/2548973/204336849-25d87b44-6f5d-487b-b583-5455f306a470.png">

(note that the `...` is because the output is too long and VSCode
truncated it)
1 year ago
Harrison Chase 261029cef3
bump version to 0.0.23 (#211) 1 year ago
Harrison Chase b94244eb12
nits (#210)
use json.dump

move test to integration tests (since it requires huggingface_hub)
1 year ago
Akash Samant ae72cf84b8
Save Prompts (#194) 1 year ago
Bagatur b90e25f786
Add HuggingFace Hub Embeddings (#125)
Add support for calling HuggingFace embedding models
using the HuggingFaceHub Inference API. New class mirrors
the existing HuggingFaceHub LLM implementation. Currently
only supports 'sentence-transformers' models.

Closes #86
1 year ago
Harrison Chase 287f1857ee
fix self ask w search (#206) 1 year ago
Harrison Chase 3eddbd11e4
bump version to 22 (#202) 1 year ago
Harrison Chase 05c5d0b8ee
add custom prompt notebooks (#198) 1 year ago
Harrison Chase fcb9b2ffe5
Harrison/agent memory (#197)
add doc for agent with memory
1 year ago
Harrison Chase 6eab5254e5
add docs for custom agents (#196) 1 year ago
Harrison Chase 08deed9002
Harrison/memory docs (#195)
update memory docs and change variables
1 year ago
Harrison Chase 199794086d
bump verion to 0.0.21 (#190) 2 years ago
Harrison Chase b913df3774
make attrs public (#187)
since they are used outside of the class, should be public
2 years ago
Harrison Chase ae9c6257fe
Harrison/arbitrary params (#186) 2 years ago
Samantha Whitmore a408ed3ea3
Samantha/add conversation chain (#166)
Add MemoryChain and ConversationChain as chains that take a docstore in
addition to the prompt, and use the docstore to stuff context into the
prompt. This can be used to have an ongoing conversation with a chatbot.

Probably needs a bit of refactoring for code quality

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2 years ago
Harrison Chase 736b6ee65c
fix search return type (#177) 2 years ago
Samantha Whitmore 09f301cd38
Add add_example method to all ExampleSelector classes, with tests (#178)
Also updated docs, and noticed an issue with the add_texts method on
VectorStores that I had missed before -- the metadatas arg should be
required to match the classmethod which initializes the VectorStores
(the add_example methods break otherwise in the ExampleSelectors)
2 years ago
Harrison Chase 1b81f3b125
bump version 0.0.20 (#174) 2 years ago
Harrison Chase 5d887970f6
change to agent (#173) 2 years ago
Harrison Chase d70b5a2cbe
Harrison/version 0019 (#172) 2 years ago
Harrison Chase d3a7429f61
(WIP) agents (#171) 2 years ago
Harrison Chase 22bd12a097
make prompt a variable in vector db qa (#170) 2 years ago
Harrison Chase 4a4dfbfbed
Harrison/sequential chains (#168)
add support for basic sequential chains
2 years ago
Harrison Chase 15c19fcc60
bump version to 0.0.18 (#167) 2 years ago
Samantha Whitmore 315b0c09c6
wip: add method for both docstore and embeddings (#119)
this will break atm but wanted to get thoughts on implementation.

1. should add() be on docstore interface?
2. should InMemoryDocstore change to take a list of documents as init?
(makes this slightly easier to implement in FAISS -- if we think it is
less clean then could expose a method to get the number of documents
currently in the dict, and perform the logic of creating the necessary
dictionary in the FAISS.add_texts method.

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2 years ago
Harrison Chase 243211a5ae
bump version to 0017 (#161) 2 years ago
Harrison Chase a19ad935b3
Harrison/verbose prompt (#159)
Add printing of prompt to LLMChain
2 years ago
Harrison Chase c02eb199b6
add few shot example (#148) 2 years ago
Harrison Chase 8869b0ab0e
bump version to 0.0.16 (#157) 2 years ago
Harrison Chase b15c84e19d
Harrison/chain lab (#156) 2 years ago
Harrison Chase 0ac08bbca6
bump version to 0.0.15 (#154) 2 years ago
Nicholas Larus-Stone 0c3ae78ec1
chore: update ascii colors to work with dark mode (#152) 2 years ago
Nicholas Larus-Stone ca4b10bb74
feat: add option to ignore or restrict to SQL tables (#151)
`SQLDatabase` now accepts two `init` arguments:
1. `ignore_tables` to pass in a list of tables to not search over
2. `include_tables` to restrict to a list of tables to consider
2 years ago
Harrison Chase d2f9288be6
add metadata to documents (#153)
add concept of metadata to document
2 years ago
Harrison Chase d775ddd749
add apply functionality (#150) 2 years ago
Harrison Chase 4f1bf159f4
bump version to 0.0.14 (#145) 2 years ago
Harrison Chase b504cd739f
Harrison/cleanup env check (#144) 2 years ago
Harrison Chase a4b502d92f
fix env var loader (#143) 2 years ago
Harrison Chase 1835e8a681
prompt nit (#141)
doing some cleanup, and i think this just simplifies things...
2 years ago
Harrison Chase bbb405a492
update colors (#140) 2 years ago
Harrison Chase 9f223e6ccc
Harrison/fix lint (#138) 2 years ago
Delip Rao 76cecf8165
A fix for Jupyter environment variable issue (#135)
- fixes the Jupyter environment variable issues mentioned in issue #134 
- fixes format/lint issues in some unrelated files (from make
format/lint)


![image](https://user-images.githubusercontent.com/347398/201599322-090af858-362d-4d69-bf59-208aea65419a.png)
2 years ago
Harrison Chase ced29b816b
remove extra run from merge conflict (#133) 2 years ago
Harrison Chase 11d37d556e
bump version 0.0.13 (#132) 2 years ago
Harrison Chase f23b3ceb49
consolidate run functions (#126)
consolidating logic for when a chain is able to run with single input
text, single output text

open to feedback on naming, logic, usefulness
2 years ago
Eugene Yurtsev 2910f50a3c
Fix a few typos and wrapped f-strings (#128)
Fix a few typos and wrapped f-strings
2 years ago
Edmar Ferreira 8a5ec894e7
Prompt from file proof of concept using plain text (#127)
This is a simple proof of concept of using external files as templates. 
I'm still feeling my way around the codebase.
As a user, I want to use files as prompts, so it will be easier to
manage and test prompts.
The future direction is to use a template engine, most likely Mako.
2 years ago
Harrison Chase d87e73ddb1
huggingface tokenizer (#75) 2 years ago
Eugene Yurtsev f5a588a165
Add py.typed marker to package (#121)
- Update
- update
2 years ago
Harrison Chase 47af2bcee4
vector db qa (#71) 2 years ago
Harrison Chase 7467243a42
bump version 0.0.12 (#116) 2 years ago
Harrison Chase e43534d41c
add integration with manifest (#62) 2 years ago
Harrison Chase 9f878e43d8
Harrison/lintai21 (#114) 2 years ago
tomeras91 d8734ce5ad
Add AI21 LLMs (#99)
Integrate AI21 /complete API into langchain, to allow access to Jurassic
models.
2 years ago
Harrison Chase 2179ea3103
remove unnecc variables (#113)
i dont think either of these variables are used?
2 years ago
Harrison Chase da445e474d
version 0.0.11 (#112) 2 years ago
Samantha Whitmore a0780cc930
OptimizedPrompt -- k-shot example choice backed by semantic search (#91) 2 years ago
Delip Rao 3ee6e332dd
Implements NLTK and Spacy-based TextSplitters (#103)
This PR is for Issue #88 

- [x] `make format`
- [x] `make lint`
- [x] `make tests`
2 years ago
issam9 28282ad099
Issam9/cohere embeddings (#105)
Add support for cohere embeddings
2 years ago
Samantha Whitmore 386a14a19f
Change NLPCloud default model (#100) 2 years ago
Harrison Chase 5b7aed34a3
bump version to 0.0.10 (#98) 2 years ago
Harrison Chase db37bd089f
model laboratory (#95) 2 years ago
Samantha Whitmore 2ddab88c06
Update VectorStore interface to contain from_texts, enforce common in… (#97)
…terface
2 years ago
Samantha Whitmore 61f12229df
Create VectorStore interface (#92) 2 years ago
Harrison Chase b9f61390e9
add text2text generation (#93)
fixes issue #90
2 years ago
Samantha Whitmore e48e562ea5
ElasticVectorSearch: Add in vector search backed by Elastic (#67)
![image](https://user-images.githubusercontent.com/6690839/200147455-33a68e20-c3c0-4045-9bff-598b38ae8fb2.png)

woo!

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2 years ago
Samantha Whitmore efbc03bda8
NLPCloud client integration (#81)
lots of kwargs! generation docs here:
https://docs.nlpcloud.com/#generation

This somewhat breaks the paradigm introduced in LLM base class as the
stop sequence isn't a list, and should rightfully be introduced at the
time of initialization of the class, along with the other kwargs that
depend on its presence (e.g. remove_end_sequence, etc.) curious if you'd
want to refactor LLM base class to take out stop as a specific named
kwarg?
2 years ago
Harrison Chase 6d8a657676
bump to version 0.0.9 (#82) 2 years ago
Harrison Chase 6cff2837bb
Harrison/fix lint (#80) 2 years ago