Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-11-06 03:20:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

Strip sitemap entries (#2132)

Browse Source 
Loading this sitemap didn't work for me
https://www.alzallies.com/sitemap.xml

Changing this fixed it and it seems like a good idea to do it in
general.

Integration tests pass
This commit is contained in:
Sebastien Kerbrat 2023-03-28 22:56:07 -07:00 committed by GitHub
parent 27f80784d0
commit 4ab66c4f52
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
langchain/document_loaders/sitemap.py
View File

@ -58,7 +58,7 @@ class SitemapLoader(WebBaseLoader):
els = self.parse_sitemap(soup)
results = self.scrape_all([el["loc"] for el in els if "loc" in el])
results = self.scrape_all([el["loc"].strip() for el in els if "loc" in el])
return [
Document(