Commit Graph

126 Commits (cea380174f738109df7467b1fc2b72a0c388f4c1)

Author SHA1 Message Date
Ankush Gola 8d0869c6d3
change run to use args and kwargs (#367)
Before, `run` was not able to be called with multiple arguments. This
expands the functionality.
2 years ago
Harrison Chase a7084ad6e4
Harrison/version 0040 (#366) 2 years ago
mrbean 50257fce59
Support Streaming Tokens from OpenAI (#364)
https://github.com/hwchase17/langchain/issues/363

@hwchase17 how much does this make you want to cry?
2 years ago
mrbean fe6695b9e7
Add HuggingFacePipeline LLM (#353)
https://github.com/hwchase17/langchain/issues/354

Add support for running your own HF pipeline locally. This would allow
you to get a lot more dynamic with what HF features and models you
support since you wouldn't be beholden to what is hosted in HF hub. You
could also do stuff with HF Optimum to quantize your models and stuff to
get pretty fast inference even running on a laptop.
2 years ago
Harrison Chase c1b50b7b13
Harrison/map reduce merge (#344)
Co-authored-by: John Nay <JohnNay@users.noreply.github.com>
2 years ago
Harrison Chase 78b31e5966
Harrison/cache (#343) 2 years ago
Harrison Chase 8cf62ce06e
Harrison/single input (#347)
allow passing of single input into chain

Co-authored-by: thepok <richterthepok@yahoo.de>
2 years ago
Harrison Chase 9bb7195085
Harrison/llm saving (#331)
Co-authored-by: Akash Samant <70665700+asamant21@users.noreply.github.com>
2 years ago
Hunter Gerlach 482611f426
unit test / code coverage improvements (#322)
This PR has two contributions:

1. Add test for when stop token is found in middle of text

2. Add code coverage tooling and instructions
- Add pytest-cov via poetry
- Add necessary config files
- Add new make instruction for `coverage`
- Update README with coverage guidance
- Update minor README formatting/spelling

Co-authored-by: Hunter Gerlach <hunter@huntergerlach.com>
2 years ago
Shobith Alva 19a9fa16a9
Add `clear()` method for `Memory` (#305)
a simple helper to clear the buffer in `Conversation*Memory` classes
2 years ago
Harrison Chase e02d6b2288
beta: logger (#307) 2 years ago
andersenchen 5267ebce2d
Add LLMCheckerChain (#281)
Implementation of https://github.com/jagilley/fact-checker. Works pretty
well.

<img width="993" alt="Screenshot 2022-12-07 at 4 41 47 PM"
src="https://user-images.githubusercontent.com/101075607/206302751-356a19ff-d000-4798-9aee-9c38b7f532b9.png">

Verifying this manually:
1. "Only two kinds of egg-laying mammals are left on the planet
today—the duck-billed platypus and the echidna, or spiny anteater."
https://www.scientificamerican.com/article/extreme-monotremes/
2. "An [Echidna] egg weighs 1.5 to 2 grams (0.05 to 0.07
oz)[[19]](https://en.wikipedia.org/wiki/Echidna#cite_note-19) and is
about 1.4 centimetres (0.55 in) long."
https://en.wikipedia.org/wiki/Echidna#:~:text=sleep%20is%20suppressed.-,Reproduction,a%20reptile%2Dlike%20egg%20tooth.
3. "A [platypus] lays one to three (usually two) small, leathery eggs
(similar to those of reptiles), about 11 mm (7⁄16 in) in diameter and
slightly rounder than bird eggs."
https://en.wikipedia.org/wiki/Platypus#:~:text=It%20lays%20one%20to%20three,slightly%20rounder%20than%20bird%20eggs.
4. Therefore, an Echidna is the mammal that lays the biggest eggs.


cc @hwchase17
2 years ago
Harrison Chase 3c1c7ba672
update branch name in gha (#274) 2 years ago
Akash Samant 48b093823e
Add a Transformation Chain (#257)
Arbitrary transformation chains that can be used to add dictionary
extractions from llms/other chains
2 years ago
coyotespike b7bef36ee1
BashChain (#260)
Love the project, a ton of fun!

I think the PR is pretty self-explanatory, happy to make any changes! I
am working on using it in an `LLMBashChain` and may update as that
progresses.

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2 years ago
Harrison Chase 28be37f470
LLMRequestsChain (#267) 2 years ago
John McDonnell 68666d6a22
Gracefully degrade when model asks for nonexistent tool (#268)
Not yet tested, but very simple change, assumption is that we're cool
with just producing a generic output when tool is not found
2 years ago
Harrison Chase f5c665a544
combine python files (#256) 2 years ago
Harrison Chase db58032973
introduce output parser (#250) 2 years ago
Harrison Chase a9ce04201f
Harrison/improve usability of api chain (#247)
improve usability of api chain
2 years ago
Harrison Chase c897bd6cbd
api chain (#246)
Co-authored-by: Subhash Ramesh <33400216+thecooltechguy@users.noreply.github.com>
2 years ago
Xupeng (Tony) Tong bb4bf9d6d0
chore: minor clean up / formatting (#233)
to get familiarize with the project
2 years ago
Harrison Chase 3ca2c8d6c5
allow passing of stop params into openai (#232) 2 years ago
Harrison Chase ca2394028f
move search to not be a chain (#226) 2 years ago
Andrew Gleave ea67c049f0
Support SQL statements that return no results (#222)
Adds support for statements such as insert, update etc which do not
return any rows.

`engine.execute` is deprecated and so execution has been updated to use
`connection.exec_driver_sql` as-per:


https://docs.sqlalchemy.org/en/14/core/connections.html#sqlalchemy.engine.Engine.execute
2 years ago
Akash Samant d368c43648
Bug Fix (#221)
Quick bug fix for semantic similarity vector injection
2 years ago
Harrison Chase 1b9b8efbc9
pal chain (#207)
from https://arxiv.org/pdf/2211.10435.pdf
2 years ago
Harrison Chase b94244eb12
nits (#210)
use json.dump

move test to integration tests (since it requires huggingface_hub)
2 years ago
Akash Samant ae72cf84b8
Save Prompts (#194) 2 years ago
Bagatur b90e25f786
Add HuggingFace Hub Embeddings (#125)
Add support for calling HuggingFace embedding models
using the HuggingFaceHub Inference API. New class mirrors
the existing HuggingFaceHub LLM implementation. Currently
only supports 'sentence-transformers' models.

Closes #86
2 years ago
Harrison Chase 6eab5254e5
add docs for custom agents (#196) 2 years ago
Harrison Chase 08deed9002
Harrison/memory docs (#195)
update memory docs and change variables
2 years ago
Harrison Chase b913df3774
make attrs public (#187)
since they are used outside of the class, should be public
2 years ago
Harrison Chase ae9c6257fe
Harrison/arbitrary params (#186) 2 years ago
Samantha Whitmore a408ed3ea3
Samantha/add conversation chain (#166)
Add MemoryChain and ConversationChain as chains that take a docstore in
addition to the prompt, and use the docstore to stuff context into the
prompt. This can be used to have an ongoing conversation with a chatbot.

Probably needs a bit of refactoring for code quality

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2 years ago
Harrison Chase 4334ffa6f9
Harrison/clean up language (#179)
dynamic prompts are no longer a thing
2 years ago
Samantha Whitmore 09f301cd38
Add add_example method to all ExampleSelector classes, with tests (#178)
Also updated docs, and noticed an issue with the add_texts method on
VectorStores that I had missed before -- the metadatas arg should be
required to match the classmethod which initializes the VectorStores
(the add_example methods break otherwise in the ExampleSelectors)
2 years ago
Harrison Chase d3a7429f61
(WIP) agents (#171) 2 years ago
Harrison Chase 4a4dfbfbed
Harrison/sequential chains (#168)
add support for basic sequential chains
2 years ago
Samantha Whitmore 315b0c09c6
wip: add method for both docstore and embeddings (#119)
this will break atm but wanted to get thoughts on implementation.

1. should add() be on docstore interface?
2. should InMemoryDocstore change to take a list of documents as init?
(makes this slightly easier to implement in FAISS -- if we think it is
less clean then could expose a method to get the number of documents
currently in the dict, and perform the logic of creating the necessary
dictionary in the FAISS.add_texts method.

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2 years ago
Harrison Chase c02eb199b6
add few shot example (#148) 2 years ago
Nicholas Larus-Stone 0c3ae78ec1
chore: update ascii colors to work with dark mode (#152) 2 years ago
Nicholas Larus-Stone ca4b10bb74
feat: add option to ignore or restrict to SQL tables (#151)
`SQLDatabase` now accepts two `init` arguments:
1. `ignore_tables` to pass in a list of tables to not search over
2. `include_tables` to restrict to a list of tables to consider
2 years ago
Harrison Chase 1835e8a681
prompt nit (#141)
doing some cleanup, and i think this just simplifies things...
2 years ago
Harrison Chase bbb405a492
update colors (#140) 2 years ago
Harrison Chase 9f223e6ccc
Harrison/fix lint (#138) 2 years ago
Delip Rao 76cecf8165
A fix for Jupyter environment variable issue (#135)
- fixes the Jupyter environment variable issues mentioned in issue #134 
- fixes format/lint issues in some unrelated files (from make
format/lint)


![image](https://user-images.githubusercontent.com/347398/201599322-090af858-362d-4d69-bf59-208aea65419a.png)
2 years ago
Harrison Chase f23b3ceb49
consolidate run functions (#126)
consolidating logic for when a chain is able to run with single input
text, single output text

open to feedback on naming, logic, usefulness
2 years ago
Edmar Ferreira 8a5ec894e7
Prompt from file proof of concept using plain text (#127)
This is a simple proof of concept of using external files as templates. 
I'm still feeling my way around the codebase.
As a user, I want to use files as prompts, so it will be easier to
manage and test prompts.
The future direction is to use a template engine, most likely Mako.
2 years ago
Harrison Chase d87e73ddb1
huggingface tokenizer (#75) 2 years ago
Harrison Chase e43534d41c
add integration with manifest (#62) 2 years ago
tomeras91 d8734ce5ad
Add AI21 LLMs (#99)
Integrate AI21 /complete API into langchain, to allow access to Jurassic
models.
2 years ago
Samantha Whitmore a0780cc930
OptimizedPrompt -- k-shot example choice backed by semantic search (#91) 2 years ago
Delip Rao 3ee6e332dd
Implements NLTK and Spacy-based TextSplitters (#103)
This PR is for Issue #88 

- [x] `make format`
- [x] `make lint`
- [x] `make tests`
2 years ago
issam9 28282ad099
Issam9/cohere embeddings (#105)
Add support for cohere embeddings
2 years ago
Delip Rao 95dd2f140e
Make Integration Tests "work" again (#106)
This fixes Issue #104 

The tests for HF Embeddings is skipped because of the segfault issue
mentioned there. Perhaps, a new issue should be created for that?
2 years ago
Harrison Chase db37bd089f
model laboratory (#95) 2 years ago
Harrison Chase b9f61390e9
add text2text generation (#93)
fixes issue #90
2 years ago
Samantha Whitmore efbc03bda8
NLPCloud client integration (#81)
lots of kwargs! generation docs here:
https://docs.nlpcloud.com/#generation

This somewhat breaks the paradigm introduced in LLM base class as the
stop sequence isn't a list, and should rightfully be introduced at the
time of initialization of the class, along with the other kwargs that
depend on its presence (e.g. remove_end_sequence, etc.) curious if you'd
want to refactor LLM base class to take out stop as a specific named
kwarg?
2 years ago
issam9 990cd821cc
Issam/hf embeddings (#68)
Add support of HuggingFace embedding models
2 years ago
Harrison Chase eb36317f9a
Harrison/fix imports (#72)
fix imports and add section to notebook
2 years ago
Samantha Whitmore a5b61d59e1
Refactor prompts into module, add example generation utils (#64) 2 years ago
Harrison Chase 2456a547de
mrkl (#42) 2 years ago
Samantha Whitmore c636488fe5
DynamicPrompt class creation (#49)
Checking that this structure looks generally ok -- going to sub in logic
where the TODO comment is then add a test.
2 years ago
Harrison Chase 4cc18d6c2a
Harrison/pretty print (#57)
make stuff look nice
2 years ago
Harrison Chase 76aff023d7
FAISS and embedding support (#48)
also adds embeddings and an in memory docstore
2 years ago
Harrison Chase e982cf4b2e
Harrison/update docstore (#47)
change docstore interface
2 years ago
Harrison Chase 160af4ba6b
Harrison/map reduce (#36) 2 years ago
Harrison Chase fba30e07d1
factor out mock python repl (#43) 2 years ago
Harrison Chase 7b0d02ac51
prompt templating (#41)
Co-authored-by: Samantha Whitmore <whitmore.samantha@gmail.com>
2 years ago
Harrison Chase af81e9ca9c
add sql database (#35) 2 years ago
Harrison Chase ce7b14b843
Harrison/add react chain (#24)
from https://arxiv.org/abs/2210.03629

still need to think if docstore abstraction makes sense
2 years ago
Harrison Chase 020c42dcae
Harrison/add huggingface hub (#23)
Add support for huggingface hub

I could not find a good way to enforce stop tokens over the huggingface
hub api - that needs to hopefully be cleaned up in the future
2 years ago
Harrison Chase d2fdcba29d
fix test name (#22) 2 years ago
Harrison Chase 1ef3ab4d0e
Harrison/add natbot (#18) 2 years ago
Harrison Chase 18aeb72012 initial commit 2 years ago