This PR makes the `"\n\n"` string with which `StuffDocumentsChain` joins
formatted documents a property so it can be configured. The new
`document_separator` property defaults to `"\n\n"` so the change is
backwards compatible.
During the import of langchain, SQLAlchemy was throeing an errror
`ImportError: cannot import name 'Mapped' from 'sqlalchemy.orm'`. This
is becaue the Mapped name was introduced in v1.4
This PR includes some minor alignment updates, including:
- metadata object extended to support contractAddress, blockchainType,
and tokenId
- notebook doc better aligned to standard langchain format
- startToken changed from int to str to support multiple hex value types
on the Alchemy API
The updated metadata will look like the below. It's possible for a
single contractAddress to exist across multiple blockchains (e.g.
Ethereum, Polygon, etc.) so it's important to include the
blockchainType.
```
metadata = {"source": self.contract_address,
"blockchain": self.blockchainType,
"tokenId": tokenId}
```
At the moment all content in Confluence is retrieved by default,
including archived content.
Often, this is undesired as the content is not relevant anymore.
**Notes**
Fetching pages by label does not support excluding archived content.
This may lead to unexpected results.
For many applications of LLM agents, the environment is real (internet,
database, REPL, etc). However, we can also define agents to interact in
simulated environments like text-based games. This is an example of how
to create a simple agent-environment interaction loop with
[Gymnasium](https://github.com/Farama-Foundation/Gymnasium) (formerly
[OpenAI Gym](https://github.com/openai/gym)).
This **partially** addresses
https://github.com/hwchase17/langchain/issues/1524, but it's also useful
for some of our use cases.
This `DocstoreFn` allows to lookup a document given a function that
accepts the `search` string without the need to implement a custom
`Docstore`.
This could be useful when:
* you don't want to implement a `Docstore` just to provide a custom
`search`
* it's expensive to construct an `InMemoryDocstore`/dict
* you retrieve documents from remote sources
* you just want to reuse existing objects
- Added links to the vectorstore providers
- Added installation code (it is not clear that we have to go to the
`LangChan Ecosystem` page to get installation instructions.)
Add other File Utilities, include
- List Directory
- Search for file
- Move
- Copy
- Remove file
Bundle as toolkit
Add a notebook that connects to the Chat Agent, which somewhat supports
multi-arg input tools
Update original read/write files to return the original dir paths and
better handle unsupported file paths.
Add unit tests
Adds a PlayWright web browser toolkit with the following tools:
- NavigateTool (navigate_browser) - navigate to a URL
- NavigateBackTool (previous_page) - wait for an element to appear
- ClickTool (click_element) - click on an element (specified by
selector)
- ExtractTextTool (extract_text) - use beautiful soup to extract text
from the current web page
- ExtractHyperlinksTool (extract_hyperlinks) - use beautiful soup to
extract hyperlinks from the current web page
- GetElementsTool (get_elements) - select elements by CSS selector
- CurrentPageTool (current_page) - get the current page URL
I think the logic of
https://github.com/hwchase17/langchain/pull/3684#pullrequestreview-1405358565
is too confusing.
I prefer this alternative because:
- All `Tool()` implementations by default will be treated the same as
before. No breaking changes.
- Less reliance on pydantic magic
- The decorator (which only is typed as returning a callable) can infer
schema and generate a structured tool
- Either way, the recommended way to create a custom tool is through
inheriting from the base tool
This notebook showcases how to implement a multi-agent simulation where
a privileged agent decides who to speak.
This follows the polar opposite selection scheme as [multi-agent
decentralized speaker
selection](https://python.langchain.com/en/latest/use_cases/agent_simulations/multiagent_bidding.html).
We show an example of this approach in the context of a fictitious
simulation of a news network. This example will showcase how we can
implement agents that
- think before speaking
- terminate the conversation
Tradeoffs here:
- No lint-time checking for compatibility
- Differs from JS package
- The signature inference, etc. in the base tool isn't simple
- The `args_schema` is optional
Pros:
- Forwards compatibility retained
- Doesn't break backwards compatibility
- User doesn't have to think about which class to subclass (single base
tool or dynamic `Tool` interface regardless of input)
- No need to change the load_tools, etc. interfaces
Co-authored-by: Hasan Patel <mangafield@gmail.com>
Resolves#3664
Next PR will be to clean up CI to catch this earlier. Triaging this, it
looks like it wasn't caught because pexpect is a `poetry` dependency.
---------
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
This catches the warning raised when using duckdb, asserts that it's as expected.
The goal is to resolve all existing warnings to make unit-testing much stricter.
Adding a lazy iteration for document loaders.
Following the plan here:
https://github.com/hwchase17/langchain/pull/2833
Keeping the `load` method as is for backwards compatibility. The `load`
returns a materialized list of documents and downstream users may rely on that
fact.
A new method that returns an iterable is introduced for handling lazy
loading.
---------
Co-authored-by: Zander Chase <130414180+vowelparrot@users.noreply.github.com>
Alternate implementation of #3452 that relies on a generic query
constructor chain and language and then has vector store-specific
translation layer. Still refactoring and updating examples but general
structure is there and seems to work s well as #3452 on exampels
---------
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
This PR introduces a Blob data type and a Blob loader interface.
This is the first of a sequence of PRs that follows this proposal:
https://github.com/hwchase17/langchain/pull/2833
The primary goals of these abstraction are:
* Decouple content loading from content parsing code.
* Help duplicated content loading code from document loaders.
* Make lazy loading a default for langchain.
### Summary
Updates the `UnstructuredURLLoader` to include a "elements" mode that
retains additional metadata from `unstructured`. This makes
`UnstructuredURLLoader` consistent with other unstructured loaders,
which also support "elements" mode. Patched mode into the existing
`UnstructuredURLLoader` class instead of inheriting from
`UnstructuredBaseLoader` because it significantly simplified the
implementation.
### Testing
This should still work and show the url in the source for the metadata
```python
from langchain.document_loaders import UnstructuredURLLoader
urls = ["https://www.understandingwar.org/sites/default/files/Russian%20Offensive%20Campaign%20Assessment%2C%20April%2011%2C%202023.pdf"]
loader = UnstructuredURLLoader(urls=urls, headers={"Accept": "application/json"}, strategy="fast")
docs = loader.load()
print(docs[0].page_content[:1000])
docs[0].metadata
```
This should now work and show additional metadata from `unstructured`.
This should still work and show the url in the source for the metadata
```python
from langchain.document_loaders import UnstructuredURLLoader
urls = ["https://www.understandingwar.org/sites/default/files/Russian%20Offensive%20Campaign%20Assessment%2C%20April%2011%2C%202023.pdf"]
loader = UnstructuredURLLoader(urls=urls, headers={"Accept": "application/json"}, strategy="fast", mode="elements")
docs = loader.load()
print(docs[0].page_content[:1000])
docs[0].metadata
```
