mirror of
https://github.com/hwchase17/langchain
synced 2024-10-29 17:07:25 +00:00
8e41143bf5
# Add generic document loader * This PR adds a generic document loader which can assemble a loader from a blob loader and a parser * Adds a registry for parsers * Populate registry with a default mimetype based parser ## Expected changes - Parsing involves loading content via IO so can be sped up via: * Threading in sync * Async - The actual parsing logic may be computatinoally involved: may need to figure out to add multi-processing support - May want to add suffix based parser since suffixes are easier to specify in comparison to mime types ## Before submitting No notebooks yet, we first need to get a few of the basic parsers up (prior to advertising the interface) |
||
|---|---|---|
| .. | ||
| agents | ||
| callbacks | ||
| chains | ||
| chat_models | ||
| client | ||
| data | ||
| docstore | ||
| document_loaders | ||
| evaluation | ||
| llms | ||
| memory | ||
| output_parsers | ||
| prompts | ||
| retrievers | ||
| tools | ||
| utilities | ||
| vectorstores | ||
| __init__.py | ||
| conftest.py | ||
| test_bash.py | ||
| test_depedencies.py | ||
| test_document_transformers.py | ||
| test_formatting.py | ||
| test_math_utils.py | ||
| test_pytest_config.py | ||
| test_python.py | ||
| test_schema.py | ||
| test_sql_database_schema.py | ||
| test_sql_database.py | ||
| test_text_splitter.py | ||