0a2ff40fcc
**Description:** Add empty string default for api_key and change `server_url` to `url` to match existing loaders. - [x] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17. |
||
---|---|---|
.. | ||
langchain_unstructured | ||
scripts | ||
tests | ||
.gitignore | ||
LICENSE | ||
Makefile | ||
poetry.lock | ||
pyproject.toml | ||
README.md |
langchain-unstructured
This package contains the LangChain integration with Unstructured
Installation
pip install -U langchain-unstructured
And you should configure credentials by setting the following environment variables:
export UNSTRUCTURED_API_KEY="your-api-key"
Loaders
Partition and load files using either the unstructured-client
sdk and the
Unstructured API or locally using the unstructured
library.
API:
To partition via the Unstructured API pip install unstructured-client
and set
partition_via_api=True
and define api_key
. If you are running the unstructured API
locally, you can change the API rule by defining url
when you initialize the
loader. The hosted Unstructured API requires an API key. See the links below to
learn more about our API offerings and get an API key.
Local:
By default the file loader uses the Unstructured partition
function and will
automatically detect the file type.
In addition to document specific partition parameters, Unstructured has a rich set of "chunking" parameters for post-processing elements into more useful text segments for uses cases such as Retrieval Augmented Generation (RAG). You can pass additional Unstructured kwargs to the loader to configure different unstructured settings.
Setup:
pip install -U langchain-unstructured
pip install -U unstructured-client
export UNSTRUCTURED_API_KEY="your-api-key"
Instantiate:
from langchain_unstructured import UnstructuredLoader
loader = UnstructuredLoader(
file_path = ["example.pdf", "fake.pdf"],
api_key=UNSTRUCTURED_API_KEY,
partition_via_api=True,
chunking_strategy="by_title",
strategy="fast",
)
Load:
docs = loader.load()
print(docs[0].page_content[:100])
print(docs[0].metadata)
References
https://docs.unstructured.io/api-reference/api-services/sdk https://docs.unstructured.io/api-reference/api-services/overview https://docs.unstructured.io/open-source/core-functionality/partitioning https://docs.unstructured.io/open-source/core-functionality/chunking