Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-11-04 06:00:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

exclude dirs in async recursive loading (#11077)

Browse Source
This commit is contained in:
Bagatur 2023-09-26 09:59:04 -07:00 committed by GitHub
parent 21199cc7b4
commit 7ee8b2d1bf
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
libs/langchain/langchain/document_loaders/recursive_url_loader.py
View File

@ -233,6 +233,7 @@ class RecursiveUrlLoader(BaseLoader):
base_url=self.url, base_url=self.url,
pattern=self.link_regex, pattern=self.link_regex,
prevent_outside=self.prevent_outside, prevent_outside=self.prevent_outside,
exclude_prefixes=self.exclude_dirs,
) )
# Recursively call the function to get the children of the children # Recursively call the function to get the children of the children