mirror of
https://github.com/hwchase17/langchain
synced 2024-10-27 21:46:30 +00:00
aead062a70
Last week I added the `PDFMinerPDFasHTMLLoader`. I am adding some example code in the notebook to serve as a tutorial for how that loader can be used to create snippets of a pdf that are structured within sections. All the other loaders only provide the `Document` objects segmented by pages but that's pretty loose given the amount of other metadata that can be extracted. With the new loader, one can leverage font-size of the text to decide when a new sections starts and can segment the text more semantically as shown in the tutorial notebook. The cell shows that we are able to find the content of entire section under **Related Work** for the example pdf which is spread across 2 pages and hence is stored as two separate documents by other loaders |
||
|---|---|---|
| .. | ||
| agents | ||
| callbacks | ||
| chains | ||
| indexes | ||
| memory | ||
| models | ||
| prompts | ||
| agents.rst | ||
| chains.rst | ||
| indexes.rst | ||
| memory.rst | ||
| models.rst | ||
| prompts.rst | ||
| state_of_the_union.txt | ||