langchain

You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

History

Eugene Yurtsev cd52433ba0 community[minor]: Add `SQLDatabaseLoader` document loader (#18281 ) - Description: A generic document loader adapter for SQLAlchemy on top of LangChain's `SQLDatabaseLoader`. - Needed by: https://github.com/crate-workbench/langchain/pull/1 - Depends on: GH-16655 - Addressed to: @baskaryan, @cbornet, @eyurtsev Hi from CrateDB again, in the same spirit like GH-16243 and GH-16244, this patch breaks out another commit from https://github.com/crate-workbench/langchain/pull/1, in order to reduce the size of this patch before submitting it, and to separate concerns. To accompany the SQLAlchemy adapter implementation, the patch includes integration tests for both SQLite and PostgreSQL. Let me know if corresponding utility resources should be added at different spots. With kind regards, Andreas. ### Software Tests ```console docker compose --file libs/community/tests/integration_tests/document_loaders/docker-compose/postgresql.yml up ``` ```console cd libs/community pip install psycopg2-binary pytest -vvv tests/integration_tests -k sqldatabase ``` ``` 14 passed ``` ![image](https://github.com/langchain-ai/langchain/assets/453543/42be233c-eb37-4c76-a830-474276e01436) --------- Co-authored-by: Andreas Motl <andreas.motl@crate.io>		7 months ago
..
blob_loaders	community[patch]: doc loaders mypy fixes (#17368 )	7 months ago
parsers	community[patch]: Skip `OpenAIWhisperParser` extremely small audio chunks to avoid api error (#11450 )	7 months ago
__init__.py	community[minor]: Add `SQLDatabaseLoader` document loader (#18281 )	7 months ago
acreom.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
airbyte.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
airbyte_json.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
airtable.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
apify_dataset.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
arcgis_loader.py	community[patch]: doc loaders mypy fixes (#17368 )	7 months ago
arxiv.py	Update arxiv.py with get_summaries_as_docs inside of Arxivloader (#14953 )	9 months ago
assemblyai.py	community[patch]: doc loaders mypy fixes (#17368 )	7 months ago
astradb.py	community[patch]: Add AstraDBLoader docstring (#17873 )	7 months ago
async_html.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
athena.py	community[patch]: document_loaders: modified athena key logic to handle s3 uris without a prefix (#17526 )	7 months ago
azlyrics.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
azure_ai_data.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
azure_blob_storage_container.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
azure_blob_storage_file.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
baiducloud_bos_directory.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
baiducloud_bos_file.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
base.py	infra: add -p to mkdir in lint steps (#17013 )	7 months ago
base_o365.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
bibtex.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
bigquery.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
bilibili.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
blackboard.py	infra: add print rule to ruff (#16221 )	7 months ago
blockchain.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
brave_search.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
browserless.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
cassandra.py	community: Fix some mypy types in cassandra doc loader (#17570 )	7 months ago
chatgpt.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
chm.py	community[patch]: docstrings (#16810 )	7 months ago
chromium.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
college_confidential.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
concurrent.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
confluence.py	infra: add print rule to ruff (#16221 )	7 months ago
conllu.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
couchbase.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
csv_loader.py	community[patch]: doc loaders mypy fixes (#17368 )	7 months ago
cube_semantic.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
datadog_logs.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
dataframe.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
diffbot.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
directory.py	community[minor]: add exclude parameter to DirectoryLoader (#17316 )	7 months ago
discord.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
doc_intelligence.py	infra: add -p to mkdir in lint steps (#17013 )	7 months ago
docugami.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
docusaurus.py	docs: docstrings `langchain_community` update (#14889 )	9 months ago
dropbox.py	infra: add print rule to ruff (#16221 )	7 months ago
duckdb_loader.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
email.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
epub.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
etherscan.py	infra: add print rule to ruff (#16221 )	7 months ago
evernote.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
excel.py	Docs: fix excel document loader typo (#15470 )	8 months ago
facebook_chat.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
fauna.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
figma.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
gcs_directory.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
gcs_file.py	fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647 )	9 months ago
generic.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
geodataframe.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
git.py	community[patch]: doc loaders mypy fixes (#17368 )	7 months ago
gitbook.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
github.py	community[patch]: Add Pagination to GitHubIssuesLoader for Efficient GitHub Issues Retrieval (#16934 )	7 months ago
google_speech_to_text.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
googledrive.py	infra: add print rule to ruff (#16221 )	7 months ago
gutenberg.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
helpers.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
hn.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
html.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
html_bs.py	fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647 )	9 months ago
hugging_face_dataset.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
hugging_face_model.py	community[minor]: add hugging_face_model document loader (#17323 )	7 months ago
ifixit.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
image.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
image_captions.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
imsdb.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
iugu.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
joplin.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
json_loader.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
lakefs.py	docs: docstrings `langchain_community` update (#14889 )	9 months ago
larksuite.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
markdown.py	corrected outdated link (#15053 )	9 months ago
mastodon.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
max_compute.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
mediawikidump.py	infra: add -p to mkdir in lint steps (#17013 )	7 months ago
merge.py	langchain[minor],community[minor]: Add async methods in BaseLoader (#16634 )	8 months ago
mhtml.py	fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647 )	9 months ago
modern_treasury.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
mongodb.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
news.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
notebook.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
notion.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
notiondb.py	community[patch]: support query filters for NotionDBLoader (#17217 )	7 months ago
nuclia.py	infra: add print rule to ruff (#16221 )	7 months ago
obs_directory.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
obs_file.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
obsidian.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
odt.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
onedrive.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
onedrive_file.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
onenote.py	infra: add print rule to ruff (#16221 )	7 months ago
open_city_data.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
org_mode.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
pdf.py	community[patch]: doc loaders mypy fixes (#17368 )	7 months ago
pebblo.py	community[patch]: Fix pwd import that is not available on windows (#17532 )	7 months ago
polars_dataframe.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
powerpoint.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
psychic.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
pubmed.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
pyspark_dataframe.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
python.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
quip.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
readthedocs.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
recursive_url_loader.py	community[patch]: doc loaders mypy fixes (#17368 )	7 months ago
reddit.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
roam.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
rocksetdb.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
rspace.py	fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647 )	9 months ago
rss.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
rst.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
rtf.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
s3_directory.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
s3_file.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
sharepoint.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
sitemap.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
slack_directory.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
snowflake_loader.py	infra: add print rule to ruff (#16221 )	7 months ago
spreedly.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
sql_database.py	community[minor]: Add `SQLDatabaseLoader` document loader (#18281 )	7 months ago
srt.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
stripe.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
surrealdb.py	community[patch]: SurrealDB fix for asyncio (#16092 )	8 months ago
telegram.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
tencent_cos_directory.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
tencent_cos_file.py	fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647 )	9 months ago
tensorflow_datasets.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
text.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
tidb.py	community[minor]: Add tidb loader support (#17788 )	7 months ago
tomarkdown.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
toml.py	infra: add print rule to ruff (#16221 )	7 months ago
trello.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
tsv.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
twitter.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
unstructured.py	community[patch]: Load list of files using UnstructuredFileLoader (#16216 )	8 months ago
url.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
url_playwright.py	community[proxy]: Enhancement/add proxy support playwrighturlloader 16751 (#16822 )	7 months ago
url_selenium.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
vsdx.py	community[minor]: New documents loader for visio files (with extension .vsdx) (#16171 )	8 months ago
weather.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
web_base.py	community[patch]: Add Cookie Support to Fetch Method (#16673 )	8 months ago
whatsapp_chat.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
wikipedia.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
word_document.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
xml.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
xorbits.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463 )	9 months ago
youtube.py	community[patch]: docstrings (#16810 )	7 months ago