Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-11-06 03:20:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
98c0b093bb
langchain/libs/community/langchain_community/document_loaders
History
maang-h fc93bed8c4
community: Fix CSVLoader columns is None (#20701) 
- **Bug code**: In
langchain_community/document_loaders/csv_loader.py:100

- **Description**: currently, when 'CSVLoader' reads the column as None
in the 'csv' file, it will report an error because the 'CSVLoader' does
not verify whether the column is of str type and does not consider how
to handle the corresponding 'row_data' when the column is' None 'in the
csv. This pr provides a solution.

- **Issue:**  Fix #20699 

- **thinking:**

1. Refer to the processing method for
'langchain_community/document_loaders/csv_loader.py:100' when **'v'**
equals'None', and apply the same method to '**k**'.
(Reference`csv.DictReader` ,**'k'** will only be None when `
len(columns) < len(number_row_data)` is established)
2. **‘k’** equals None only holds when it is the last column, and its
corresponding **'v'** type is a list. Therefore, I referred to the data
format in 'Document' and used ',' to concatenated the elements in the
list.(But I'm not sure if you accept this form, if you have any other
ideas, communicate)

---------

Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>
2024-05-22 12:57:46 -07:00
..
blob_loaders langchain[patch]: Migrate document loaders to use optional langchain community imports (#21095) 2024-05-01 11:26:25 -04:00
parsers community[patch]: Fix remaining __inits__ in community (#22037) 2024-05-22 17:42:17 +00:00
__init__.py community[minor]: Oraclevs integration (#21123) 2024-05-04 03:15:35 +00:00
acreom.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
airbyte_json.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
airbyte.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
airtable.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
apify_dataset.py community[patch]: update apify integration to attribute API activity to langchain (#21909) 2024-05-20 14:49:23 -07:00
arcgis_loader.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
arxiv.py community[minor]: Implement lazy_load() for ArxivLoader (#18664) 2024-03-06 09:16:49 -05:00
assemblyai.py community[patch]: docstrings update (#20301) 2024-04-11 16:23:27 -04:00
astradb.py (all): update removal in deprecation warnings from 0.2 to 0.3 (#21265) 2024-05-03 14:29:36 -04:00
async_html.py community[minor]: allow enabling proxy in aiohttp session in AsyncHTML (#19499) 2024-05-22 18:25:06 +00:00
athena.py community[minor]: import fix (#20995) 2024-04-29 10:32:50 -04:00
azlyrics.py
azure_ai_data.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
azure_blob_storage_container.py community[patch]: type ignore fixes (#18395) 2024-03-01 11:21:02 -08:00
azure_blob_storage_file.py
baiducloud_bos_directory.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
baiducloud_bos_file.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
base_o365.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
base.py core: Move document loader interfaces to core (#17723) 2024-03-06 13:59:00 -05:00
bibtex.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
bigquery.py (all): update removal in deprecation warnings from 0.2 to 0.3 (#21265) 2024-05-03 14:29:36 -04:00
bilibili.py community[patch]: docstrings update (#20301) 2024-04-11 16:23:27 -04:00
blackboard.py community[patch]: type ignore fixes (#18395) 2024-03-01 11:21:02 -08:00
blockchain.py
brave_search.py
browserbase.py community: updated Browserbase loader (#21757) 2024-05-16 08:21:23 -07:00
browserless.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
cassandra.py [community]: Add missing docstring param to CassandraLoader (#21611) 2024-05-13 16:03:18 +00:00
chatgpt.py
chm.py community[patch]: docstrings (#16810) 2024-02-09 12:48:57 -08:00
chromium.py community[patch]: Update comments for lazy_load method (#21063) 2024-05-01 01:20:57 -04:00
college_confidential.py
concurrent.py community[patch]: import flattening fix (#20110) 2024-04-10 13:01:19 -04:00
confluence.py multiple: Remove unnecessary Ruff suppression comments (#21050) 2024-04-30 17:13:48 +00:00
conllu.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
couchbase.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
csv_loader.py community: Fix CSVLoader columns is None (#20701) 2024-05-22 12:57:46 -07:00
cube_semantic.py community[patch]: Implement lazy_load() for CubeSemanticLoader (#18535) 2024-03-05 17:32:31 -08:00
datadog_logs.py
dataframe.py community[patch]: support modin document loader (#18866) 2024-03-10 18:40:04 -07:00
diffbot.py
directory.py community: fix DirectoryLoader progress bar (#19821) 2024-04-17 21:12:16 +00:00
discord.py
doc_intelligence.py docs: community docstring updates (#21040) 2024-04-29 17:40:23 -04:00
docugami.py (all): update removal in deprecation warnings from 0.2 to 0.3 (#21265) 2024-05-03 14:29:36 -04:00
docusaurus.py
dropbox.py infra: add print rule to ruff (#16221) 2024-02-09 16:13:30 -08:00
duckdb_loader.py
email.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
epub.py
etherscan.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
evernote.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
excel.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
facebook_chat.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
fauna.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
figma.py
firecrawl.py multiple: Remove unnecessary Ruff suppression comments (#21050) 2024-04-30 17:13:48 +00:00
gcs_directory.py (all): update removal in deprecation warnings from 0.2 to 0.3 (#21265) 2024-05-03 14:29:36 -04:00
gcs_file.py (all): update removal in deprecation warnings from 0.2 to 0.3 (#21265) 2024-05-03 14:29:36 -04:00
generic.py community[patch]: import flattening fix (#20110) 2024-04-10 13:01:19 -04:00
geodataframe.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
git.py Merge pull request #18539 2024-03-06 13:25:14 -05:00
gitbook.py community[minor]: Implement lazy_load() for GitbookLoader (#18670) 2024-03-06 09:14:36 -05:00
github.py community: Implement lazy_load() for GithubFileLoader (#18584) 2024-03-05 09:35:50 -08:00
glue_catalog.py community[minor]: Add glue catalog loader (#20220) 2024-04-16 11:39:23 -04:00
google_speech_to_text.py (all): update removal in deprecation warnings from 0.2 to 0.3 (#21265) 2024-05-03 14:29:36 -04:00
googledrive.py (all): update removal in deprecation warnings from 0.2 to 0.3 (#21265) 2024-05-03 14:29:36 -04:00
gutenberg.py
helpers.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
hn.py
html_bs.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
html.py
hugging_face_dataset.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
hugging_face_model.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
ifixit.py
image_captions.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
image.py
imsdb.py
iugu.py
joplin.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
json_loader.py multiple: Remove unnecessary Ruff suppression comments (#21050) 2024-04-30 17:13:48 +00:00
kinetica_loader.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
lakefs.py
larksuite.py community[minor]: Add LarkSuite wiki document loader. (#21016) 2024-04-29 10:37:50 -04:00
llmsherpa.py community[minor]: add support for llmsherpa (#19741) 2024-03-29 16:04:57 -07:00
markdown.py
mastodon.py Merge pull request #18671 2024-03-06 13:23:14 -05:00
max_compute.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
mediawikidump.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
merge.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
mhtml.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
mintbase.py community[minor]: add mintbase loader to langchain (#20089) 2024-04-30 04:11:56 +00:00
modern_treasury.py
mongodb.py community[minor]: added a feature to filter documents in Mongoloader (#18253) 2024-03-08 12:06:35 -08:00
news.py multiple: Remove unnecessary Ruff suppression comments (#21050) 2024-04-30 17:13:48 +00:00
notebook.py community[patch]: add NotebookLoader unit test (#17721) 2024-03-29 00:27:46 +00:00
notion.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
notiondb.py community[patch]: Fix NotionDBLoader 400 Error by conditionally adding filter parameter (#19075) 2024-03-14 13:56:57 +00:00
nuclia.py infra: add print rule to ruff (#16221) 2024-02-09 16:13:30 -08:00
obs_directory.py
obs_file.py
obsidian.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
odt.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
onedrive_file.py
onedrive.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
onenote.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
open_city_data.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
oracleadb_loader.py community[minor]: add oracle autonomous database doc loader integration (#19536) 2024-03-26 17:02:18 -07:00
oracleai.py community[minor]: Oraclevs integration (#21123) 2024-05-04 03:15:35 +00:00
org_mode.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
pdf.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
pebblo.py community[patch]: Remove redundant pebblo cloud api call (#21589) 2024-05-20 17:15:16 -07:00
polars_dataframe.py
powerpoint.py
psychic.py multiple: Remove unnecessary Ruff suppression comments (#21050) 2024-04-30 17:13:48 +00:00
pubmed.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
pyspark_dataframe.py
python.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
quip.py
readthedocs.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
recursive_url_loader.py community[patch]: Using the right encoding to parse the web page in RecursiveUrlLoader (#20632) 2024-04-30 18:41:36 +00:00
reddit.py
roam.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
rocksetdb.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
rspace.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
rss.py multiple: Remove unnecessary Ruff suppression comments (#21050) 2024-04-30 17:13:48 +00:00
rst.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
rtf.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
s3_directory.py community[patch]: Skip nested directories when using S3DirectoryLoader (#17829) 2024-03-08 16:50:58 -08:00
s3_file.py community[patch]: support unstructured_kwargs for s3 loader (#15473) 2024-03-27 22:03:48 +00:00
sharepoint.py community[patch]: Changes to base_o365 and sharepoint document loaders (#20373) 2024-04-17 00:36:15 +00:00
sitemap.py community[minor]: Implement lazy_load() for SitemapLoader (#18667) 2024-03-06 09:15:35 -05:00
slack_directory.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
snowflake_loader.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
spider.py doc list not empty (#21208) 2024-05-20 08:24:06 -07:00
spreedly.py
sql_database.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
srt.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
stripe.py
surrealdb.py community[patch]: SurrealDB fix for asyncio (#16092) 2024-01-23 19:46:19 -08:00
telegram.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
tencent_cos_directory.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
tencent_cos_file.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
tensorflow_datasets.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
text.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
tidb.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
tomarkdown.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
toml.py community: Use default load() implementation in doc loaders (#18385) 2024-03-01 14:46:52 -05:00
trello.py community: Implement lazy_load() for TrelloLoader (#18658) 2024-03-06 13:04:36 -05:00
tsv.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
twitter.py
unstructured.py community[minor]: import fix (#20995) 2024-04-29 10:32:50 -04:00
url_playwright.py docs: community docstring updates (#21040) 2024-04-29 17:40:23 -04:00
url_selenium.py
url.py
vsdx.py community[patch]: import flattening fix (#20110) 2024-04-10 13:01:19 -04:00
weather.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
web_base.py community[patch]: raise_for_status logic missing in async _fetch of WebBaseLoader (#21948) 2024-05-21 23:51:03 +00:00
whatsapp_chat.py community: Implement lazy_load() for WhatsAppChatLoader (#18677) 2024-03-06 13:03:46 -05:00
wikipedia.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
word_document.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
xml.py community: better support of pathlib paths in document loaders (#18396) 2024-03-26 11:51:52 -04:00
xorbits.py
youtube.py community[patch]: docstrings (#16810) 2024-02-09 12:48:57 -08:00
yuque.py community[minor]: add Yuque document loader (#17924) 2024-03-05 15:54:07 -08:00