5 Commits (0561162dd295b989a54bb1ecf966b0e3d7823864)

Author	SHA1	Message	Date
Matt Robinson	63aa28e2a6	feat: allow the unstructured kwargs to be passed in to Unstructured document loaders (#1667) ... ### Summary Allows users to pass in `**unstructured_kwargs` to Unstructured document loaders. Implemented with the `strategy` kwargs in mind, but will pass in other kwargs like `include_page_breaks` as well. The two currently supported strategies are `"hi_res"`, which is more accurate but takes longer, and `"fast"`, which processes faster but with lower accuracy. The `"hi_res"` strategy is the default. For PDFs, if `detectron2` is not available and the user selects `"hi_res"`, the loader will fallback to using the `"fast"` strategy. ### Testing #### Make sure the `strategy` kwarg works Run the following in iPython to verify that the `"fast"` strategy is indeed faster. ```python from langchain.document_loaders import UnstructuredFileLoader loader = UnstructuredFileLoader("layout-parser-paper-fast.pdf", strategy="fast", mode="elements") %timeit loader.load() loader = UnstructuredFileLoader("layout-parser-paper-fast.pdf", mode="elements") %timeit loader.load() ``` On my system I get: ```python In [3]: from langchain.document_loaders import UnstructuredFileLoader In [4]: loader = UnstructuredFileLoader("layout-parser-paper-fast.pdf", strategy="fast", mode="elements") In [5]: %timeit loader.load() 247 ms ± 369 µs per loop (mean ± std. dev. of 7 runs, 1 loop each) In [6]: loader = UnstructuredFileLoader("layout-parser-paper-fast.pdf", mode="elements") In [7]: %timeit loader.load() 2.45 s ± 31 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) ``` #### Make sure older versions of `unstructured` still work Run `pip install unstructured==0.5.3` and then verify the following runs without error: ```python from langchain.document_loaders import UnstructuredFileLoader loader = UnstructuredFileLoader("layout-parser-paper-fast.pdf", mode="elements") loader.load() ```	1 year ago
Matt Robinson	3d5f56a8a1	docs: add quotes to `unstructured[local-inference]` install instructions (#1208) ... ### Summary Corrects the install instruction for local inference to `pip install "unstructured[local-inference]"`	1 year ago
cragwolfe	05d8969c79	Unstructured example notebook: add a pdf, related deps (#1011) ... Updates the Unstructured example notebook with a PDF example. Includes additional dependencies for PDF processing (and images, etc).	1 year ago
Harrison Chase	0998577dfe	Harrison/unstructured structured (#1004)	1 year ago
Harrison Chase	53d56d7650	Harrison/unstructured support (#903)	1 year ago