mirror of https://github.com/hwchase17/langchain
Fix inconsistent behavior of `CharacterTextSplitter` when changing `keep_separator` (#7263)
- Description: - When `keep_separator` is `True` the `_split_text_with_regex()` method in `text_splitter` uses regex to split, but when `keep_separator` is `False` it uses `str.split()`. This causes problems when the separator is a special regex character like `.` or `*`. This PR fixes that by using `re.split()` in both cases. - Issue: #7262 - Tag maintainer: @baskaryanpull/6927/head^2
parent
b151d4257a
commit
0c7a5cb206
Loading…
Reference in New Issue