Commit Graph

1144 Commits

Author SHA1 Message Date
Saurabh Misra
c33e055f17
Improve ConversationKGMemory and its function load_memory_variables (#1999)
1. Removed the `summaries` dictionary in favor of directly appending to
the summary_strings list, which avoids the unnecessary double-loop.
2. Simplified the logic for populating the `context` variable.

Co-created with GPT-4 @agihouse
2023-03-28 15:19:48 -07:00
Harrison Chase
a5bf8c9b9d
Harrison/aleph alpha embeddings (#2117)
Co-authored-by: Piotr Mazurek <piotr635@gmail.com>
Co-authored-by: PiotrMazurek <piotr.mazurek@aleph-alpha.com>
2023-03-28 15:18:03 -07:00
Nick
0874872dee
add token reduction to ConversationalRetrievalChain (#2075)
This worked for me, but I'm not sure if its the right way to approach
something like this, so I'm open to suggestions.

Adds class properties `reduce_k_below_max_tokens: bool` and
`max_tokens_limit: int` to the `ConversationalRetrievalChain`. The code
is basically copied from
[`RetreivalQAWithSourcesChain`](46d141c6cb/langchain/chains/qa_with_sources/retrieval.py (L24))
2023-03-28 15:07:31 -07:00
Alex Telon
ef25904ecb
Fixed 1 missing line in getting_started.md (#2107)
Seems like a copy paste error. The very next example does have this
line.

Please tell me if I missed something in the process and should have
created an issue or something first!
2023-03-28 15:03:28 -07:00
Francis Felici
9d6f649ba5
fix typo in docs (#2115)
simple typo
2023-03-28 15:03:17 -07:00
Harrison Chase
c58932e8fd
Harrison/better async (#2112)
Co-authored-by: Ammar Husain <ammo700@gmail.com>
2023-03-28 13:28:04 -07:00
Harrison Chase
6e85cbcce3
Harrison/unstructured validation (#2111)
Co-authored-by: kravetsmic <79907559+kravetsmic@users.noreply.github.com>
2023-03-28 13:27:52 -07:00
Tim Asp
b25dbcb5b3
add missing source field to pymupdf output (#2110)
To be consistent with other loaders for use with the `Sources` vector
workflows.
2023-03-28 13:22:05 -07:00
Harrison Chase
a554e94a1a
v125 (#2109)
for hackathon tonight!
2023-03-28 13:12:41 -07:00
Michael Gokhman
5f34dffedc
fix(llms): update default AI21 model to j2, as j1 being deprecated (#2108)
the j1-* models are marked as [Legacy] in the docs and are expected to
be deprecated in 2023-06-01 according to
https://docs.ai21.com/docs/jurassic-1-models-legacy

ensured `tests/integration_tests/llms/test_ai21.py` pass.

empirically observed that `j2-jumbo-instruct` works better the
`j2-jumbo` in various simple agent chains, as also expected given the
prompt templates are mostly zero shot.

Co-authored-by: Michael Gokhman <michaelg@ai21.com>
2023-03-28 13:07:05 -07:00
Honkware
aff33d52c5
Add OpenWeatherMap API Tool (#2083)
Added tool for OpenWeatherMap API
2023-03-28 12:02:14 -07:00
Charlie Holtz
f16c1fb6df
Add replicate take 2 (#2077)
This PR adds a replicate integration to langchain. 

It's an updated version of
https://github.com/hwchase17/langchain/pull/1993, but with updates to
match latest replicate-python code.
https://github.com/replicate/replicate-python.

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
Co-authored-by: Zeke Sikelianos <zeke@sikelianos.com>
2023-03-28 11:56:57 -07:00
Harrison Chase
a9e1043673
bump version 124 (#2101) 2023-03-28 08:58:52 -07:00
Harrison Chase
f281033362
rm pandas dependency (#2102) 2023-03-28 08:38:19 -07:00
Harrison Chase
410bf37fb8
Harrison/big query (#2100)
Co-authored-by: lu-cashmoney <lucas.corley@gmail.com>
2023-03-28 08:17:22 -07:00
Harrison Chase
eff5eed719
Harrison/jina (#2043)
Co-authored-by: numb3r3 <wangfelix87@gmail.com>
Co-authored-by: felix-wang <35718120+numb3r3@users.noreply.github.com>
2023-03-28 08:16:17 -07:00
Klein Tahiraj
d0a56f47ee
add ConversationalChatAgent to agent.__init__ (fix #2093) (#2098)
As pointed out in #2093, ConversationalChatAgent was missing from
agent.__init__. This PR fixes that.
2023-03-28 08:14:21 -07:00
Harrison Chase
9e74df2404
Fix issue#1645: Parse llm_output even there's newline (#2092) (#2099)
Fix issue#1645: Parse either whitespace or newline after 'Action Input:'
in llm_output in mrkl agent.
Unittests added accordingly.

Co-authored-by: ₿ingnan.ΞTH <brillliantz@outlook.com>
2023-03-28 08:14:09 -07:00
Stéphane Busso
0bee219cb3
feat: Add Notion database document loader (#2056)
This PR adds Notion DB loader for langchain. 

It reads content from pages within a Notion Database. It uses the Notion
API to query the database and read the pages. It also reads the metadata
from the pages and stores it in the Document object.
2023-03-28 08:07:09 -07:00
Harrison Chase
923a7dde5a
Harrison/llama index loader (#2097)
Co-authored-by: Jerry Liu <jerryjliu98@gmail.com>
2023-03-28 08:06:27 -07:00
Harrison Chase
4cd5cf2e95
notebook for tokens (#2086) 2023-03-28 07:59:40 -07:00
33ebb05251
include the tool name for on_tool_end callback (#2000)
This is useful if you rely on the `on_tool_end` callback to detect which
tool has finished in a multi agents scenario.

For example, I'm working on a project where I consume the `on_tool_end`
event where the event could be emitted by many agents or tools. Right
now the only way to know which tool has finished would be set a marker
on the `on_tool_start` and catch it on `on_tool_end`.

I didn't want to break the signature of the function, but what would
have been cleaner would be to pass the same details as in
`on_tool_start`

Co-authored-by: blob42 <spike@w530>
2023-03-28 10:23:04 -04:00
Clark
e0331b55bb
fix(sql_database): related to #2020 (#2021)
Fixed https://github.com/hwchase17/langchain/issues/2020

Co-authored-by: qianjun.wqj <qianjun.wqj@alibaba-inc.com>
2023-03-27 23:45:50 -07:00
Harrison Chase
d5825bd3e8
Harrison/whatsapp loader (#2085)
Co-authored-by: Moshe <hello@moshemalka.me>
2023-03-27 23:43:45 -07:00
iocuydi
e8d9cbca3f
Add prompt and completion token tracking (#2080)
Tracking the breakdown of token usage is useful when using GPT-4, where
prompt and completion tokens are priced differently.
2023-03-27 23:41:25 -07:00
Michael Gokhman
b5020c7d9c
docs: fix promptlayer link typo (#2005)
tiny typo, just stumbled upon it when reading the docs

Co-authored-by: Michael Gokhman <michaelg@ai21.com>
2023-03-27 23:35:54 -07:00
Deepankar Mahapatro
5bea731fb4
docs(deployment): add langchain-serve (#2006)
Adds documentation to deploy Langchain Chains & Agents using Jina.

Repo: https://github.com/jina-ai/langchain-serve
2023-03-27 23:32:04 -07:00
Harrison Chase
0e3b0c827e
Harrison/ai plugin (#2084)
Co-authored-by: Xupeng (Tony) Tong <tongxupeng.cpu@gmail.com>
2023-03-27 23:31:53 -07:00
Harrison Chase
365669a7fd
Harrison/fix save context (#2082)
Co-authored-by: Saurabh Misra <misra.saurabh1@gmail.com>
2023-03-27 23:10:46 -07:00
b7f392fdd6
[agent_executor] convenience func: lookup tool by name (#2001)
A quick convenience function to lookup a tool by name

Co-authored-by: blob42 <spike@w530>
2023-03-27 23:10:34 -07:00
Ace Eldeib
4be2f9d75a
fix: numerous broken documentation links (#2070)
seems linkchecker isn't catching them because it runs on generated html.
at that point the links are already missing.
the generation process seems to strip invalid references when they can't
be re-written from md to html.

I used https://github.com/tcort/markdown-link-check to check the doc
source directly.

There are a few false positives on localhost for development.
2023-03-27 23:07:03 -07:00
Harrison Chase
f74a1bebf5
Harrison/duckdb (#2064)
Co-authored-by: Trent Hauck <trent@trenthauck.com>
2023-03-27 19:51:34 -07:00
Harrison Chase
76ecca4d53
redis retriever (#2060) 2023-03-27 19:51:23 -07:00
Ankush Gola
b7ebb8fe30
enable streaming in anthropic llm wrapper (#2065) 2023-03-27 20:25:00 -04:00
Francisco Ingham
41c8a42e22
Improve chat tool prompt (#1989)
I have found that when the user has not asked an explicit question the
agent might have trouble answering the latest comment and might instead
try to answer a question that came before in the conversation which
would not be what is desired.

I also found that the agent might get confused with the current prompt
and talk about the tools themselves instead of the results obtained from
them.

I added two changes to the tool prompt so that the agent answers only
the last comment/question and only returns information from tool
results.
2023-03-27 16:34:01 -07:00
Francisco Ingham
1cc9e90041
Solve small bug in the kg prompt (#1988)
I think that the 'Person' line should be under 'Last line of
conversation' as is the case in the other examples in the kg prompt
2023-03-27 16:33:26 -07:00
Harrison Chase
30e3b31b04
Harrison/document cleanup (#2062)
Co-authored-by: Delip Rao <delip@users.noreply.github.com>
2023-03-27 16:32:55 -07:00
Harrison Chase
a0cd6672aa
Harrison/site map (#2061)
Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>
2023-03-27 16:28:08 -07:00
Arttii
8b5a43d720
Correctly pass filter down to the similarity_search_with_score function for chroma filtering logic (#1934)
Should slightly fix the work in #1869
2023-03-27 15:50:46 -07:00
Jonathan Pedoeem
725b668aef
Updating PromptLayer request in PromptLayer Models to be async in agenerate (#2058)
Currently in agenerate, the PromptLayer request is blocking and should
be make async. In this PR we update all of that in order to work as it
should
2023-03-27 15:24:53 -07:00
Peter Shi
024efb09f8
feat: add function similarity_search_limit_score to vectorstores.redis (#1950)
# Description
***
Add function similarity_search_limit_score and
similarity_search_with_score

# How to use
***
``
rds = Redis.from_existing_index(embeddings,
redis_url="redis://localhost:6379", index_name='link')

rds.similarity_search_limit_score(query, k=3, score=0.2)

rds.similarity_search_with_score(query, k=3)
``

---------

Co-authored-by: Peter <peter.shi@alephf.com>
2023-03-27 15:05:09 -07:00
Rajat Saxena
953e58d004
similarity_search is not accepting filters (#1964)
I have changed the name of the argument from `where` to `filter` which
is expected by `similarity_search_with_score`.

Fixes #1838

---------

Co-authored-by: Rajat Saxena <hi@rajatsaxena.dev>
2023-03-27 15:04:53 -07:00
Gerard Hernandez
f257b08406
Removed duplicate "revision_request" in constitutional_ai/prompts.py (#2046)
Removed a duplicate "revision_request" in the second example within
[this
file](https://github.com/hwchase17/langchain/blob/master/langchain/chains/constitutional_ai/prompts.py).
2023-03-27 15:04:23 -07:00
Krulknul
5e91928607
Added .as_retriever() to from_llm() calls (#2051) 2023-03-27 15:04:03 -07:00
Harrison Chase
880a6a3db5
Harrison/redis id key (#2057)
Co-authored-by: Fabrizio Ruocco <ruoccofabrizio@gmail.com>
2023-03-27 15:03:51 -07:00
cragwolfe
71e8eaff2b
UnstructuredURLLoader: allow url failures, keep processing (#1954)
By default, UnstructuredURLLoader now continues processing remaining
`urls` if encountering an error for a particular url.

If failure of the entire loader is desired as was previously the case,
use `continue_on_failure=False`.

E.g., this fails splendidly, courtesy of the 2nd url:

```
from langchain.document_loaders import UnstructuredURLLoader
urls = [
    "https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-8-2023",
    "https://doesnotexistithinkprobablynotverynotlikely.io",
    "https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-9-2023",
]
loader = UnstructuredURLLoader(urls=urls, continue_on_failure=False)
data = loader.load()
```

Issue: https://github.com/hwchase17/langchain/issues/1939
2023-03-27 14:34:14 -07:00
Daniel Chalef
6598beacdb
PydanticOutputParser unit test (#2047)
Unit test for PydanticOutputParser

---------

Co-authored-by: Daniel Chalef <daniel.chalef@private.org>
2023-03-27 14:32:56 -07:00
William FH
e4f15e4eac
Add support for YAML Spec Plugins (#2054)
It's common to use `yaml` for an OpenAPI spec used in the GPT plugins. 

For example: https://www.joinmilo.com/openapi.yaml or
https://api.slack.com/specs/openapi/ai-plugin.yaml (from [Wong2's
ChatGPT Plugins List](https://github.com/wong2/chatgpt-plugins))
2023-03-27 14:27:48 -07:00
weiyang
e50c1ea7fb
Fix the parameter error of 'Qdrant.maximal_marginal_relevance' (#1921)
Hi, first and foremost, I would like to express my gratitude for your
outstanding work; it's truly remarkable!


https://github.com/hwchase17/langchain/blob/master/langchain/vectorstores/qdrant.py#L134
It appears that there might be a minor issue with the `limit` parameter
being passed incorrectly in the `Qdrant.maximal_marginal_relevance`
function. This seems to be a typographical error.

Signed-off-by: weiyang <weiyang.ones@gmail.com>
2023-03-27 08:29:07 -07:00
goka
62e08f80de
feat #1915 support for google custom search site restricted api (#1920)
#1915 

https://developers.google.com/custom-search/v1/site_restricted_api

It is possible to search unrestricted to specific sites.
2023-03-27 08:28:55 -07:00