Commit Graph

5 Commits (432421ffa50c44e660b902adb48db40add9621d5)

Author SHA1 Message Date
Harrison Chase eda69b13f3
openai embeddings (#3488) 1 year ago
Harrison Chase c844d1fd46
Harrison/chunk size (#1549)
Co-authored-by: Florian Leuerer <31259070+floleuerer@users.noreply.github.com>
2 years ago
Hasegawa Yuya e08961ab25
Fixed openai embeddings to be safe by batching them based on token size calculation. (#991)
I modified the logic of the batch calculation for embedding according to
this cookbook

https://github.com/openai/openai-cookbook/blob/main/examples/Embedding_long_inputs.ipynb
2 years ago
Harrison Chase 91c6cea227
Harrison/batch embeds (#972)
Co-authored-by: John Dagdelen <jdagdelen@users.noreply.github.com>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
2 years ago
Harrison Chase 76aff023d7
FAISS and embedding support (#48)
also adds embeddings and an in memory docstore
2 years ago