langchain/libs/text-splitters/tests
Jiejun Tan c8c67dde6f
text-splitters[patch]: Fix HTMLSectionSplitter (#22812)
Update former pull request:
https://github.com/langchain-ai/langchain/pull/22654.

Modified `langchain_text_splitters.HTMLSectionSplitter`, where in the
latest version `dict` data structure is used to store sections from a
html document, in function `split_html_by_headers`. The header/section
element names serve as dict keys. This can be a problem when duplicate
header/section element names are present in a single html document.
Latter ones can replace former ones with the same name. Therefore some
contents can be miss after html text splitting is conducted.

Using a list to store sections can hopefully solve the problem. A Unit
test considering duplicate header names has been added.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2024-06-14 22:40:39 +00:00
..
integration_tests
test_data text-splitters[patch]: fix HTMLSectionSplitter parsing of xslt paths (#22176) 2024-06-03 20:26:59 +00:00
unit_tests text-splitters[patch]: Fix HTMLSectionSplitter (#22812) 2024-06-14 22:40:39 +00:00
__init__.py