Commit Graph

8 Commits (e257380debb8640268d2d2577f89139b3ea0b46f)

Author SHA1 Message Date
Roma 2b4e9a3efa
Add unit test for _merge_splits function (#3513)
This commit adds a new unit test for the _merge_splits function in the
text splitter. The new test verifies that the function merges text into
chunks of the correct size and overlap, using a specified separator. The
test passes on the current implementation of the function.
1 year ago
Harrison Chase f95d551f7a
Harrison/shallow metadata (#1599)
Co-authored-by: Jesse Zhang <jessetanzhang@gmail.com>
2 years ago
Harrison Chase 064741db58
Harrison/fix text splitter (#1511)
Co-authored-by: ajaysolanky <ajsolanky@gmail.com>
Co-authored-by: Ajay Solanky <ajaysolanky@saw-l14668307kd.myfiosgateway.com>
2 years ago
Harrison Chase 1511606799
Harrison/fix splitting (#563)
fix issue where text splitting could possibly create empty docs
2 years ago
Harrison Chase 1192cc0767
smart text splitter (#530)
smart text splitter that iteratively tries different separators until it
works!
2 years ago
Harrison Chase c104d507bf
Harrison/improve data augmented generation docs (#390)
Co-authored-by: cameronccohen <cameron.c.cohen@gmail.com>
Co-authored-by: Cameron Cohen <cameron.cohen@quantco.com>
2 years ago
Harrison Chase e7b625fe03
fix text splitter (#375) 2 years ago
Harrison Chase 160af4ba6b
Harrison/map reduce (#36) 2 years ago