langchain/libs/partners/unstructured
John 0a2ff40fcc
partners/unstructured: fix client api_url (#24680)
**Description:** Add empty string default for api_key and change
`server_url` to `url` to match existing loaders.

- [x] **Lint and test**: Run `make format`, `make lint` and `make test`
from the root of the package(s) you've modified. See contribution
guidelines for more: https://python.langchain.com/docs/contributing/

Additional guidelines:
- Make sure optional dependencies are imported within a function.
- Please do not add dependencies to pyproject.toml files (even optional
ones) unless they are required for unit tests.
- Most PRs should not touch more than one package.
- Changes should be backwards compatible.
- If you are adding something to community, do not re-import it in
langchain.

If no one reviews your PR within a few days, please @-mention one of
baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.
2024-07-29 11:16:41 -07:00
..
langchain_unstructured partners/unstructured: fix client api_url (#24680) 2024-07-29 11:16:41 -07:00
scripts unstructured, community, initialize langchain-unstructured package (#22779) 2024-07-24 23:21:20 +00:00
tests partners/unstructured: fix client api_url (#24680) 2024-07-29 11:16:41 -07:00
.gitignore unstructured, community, initialize langchain-unstructured package (#22779) 2024-07-24 23:21:20 +00:00
LICENSE unstructured, community, initialize langchain-unstructured package (#22779) 2024-07-24 23:21:20 +00:00
Makefile unstructured, community, initialize langchain-unstructured package (#22779) 2024-07-24 23:21:20 +00:00
poetry.lock unstructured, community, initialize langchain-unstructured package (#22779) 2024-07-24 23:21:20 +00:00
pyproject.toml partners/unstructured: fix client api_url (#24680) 2024-07-29 11:16:41 -07:00
README.md unstructured, community, initialize langchain-unstructured package (#22779) 2024-07-24 23:21:20 +00:00

langchain-unstructured

This package contains the LangChain integration with Unstructured

Installation

pip install -U langchain-unstructured

And you should configure credentials by setting the following environment variables:

export UNSTRUCTURED_API_KEY="your-api-key"

Loaders

Partition and load files using either the unstructured-client sdk and the Unstructured API or locally using the unstructured library.

API: To partition via the Unstructured API pip install unstructured-client and set partition_via_api=True and define api_key. If you are running the unstructured API locally, you can change the API rule by defining url when you initialize the loader. The hosted Unstructured API requires an API key. See the links below to learn more about our API offerings and get an API key.

Local: By default the file loader uses the Unstructured partition function and will automatically detect the file type.

In addition to document specific partition parameters, Unstructured has a rich set of "chunking" parameters for post-processing elements into more useful text segments for uses cases such as Retrieval Augmented Generation (RAG). You can pass additional Unstructured kwargs to the loader to configure different unstructured settings.

Setup:

    pip install -U langchain-unstructured
    pip install -U unstructured-client
    export UNSTRUCTURED_API_KEY="your-api-key"

Instantiate:

from langchain_unstructured import UnstructuredLoader

loader = UnstructuredLoader(
    file_path = ["example.pdf", "fake.pdf"],
    api_key=UNSTRUCTURED_API_KEY,
    partition_via_api=True,
    chunking_strategy="by_title",
    strategy="fast",
)

Load:

docs = loader.load()

print(docs[0].page_content[:100])
print(docs[0].metadata)

References

https://docs.unstructured.io/api-reference/api-services/sdk https://docs.unstructured.io/api-reference/api-services/overview https://docs.unstructured.io/open-source/core-functionality/partitioning https://docs.unstructured.io/open-source/core-functionality/chunking