3 Commits

Author	SHA1	Message	Date
Taqi Jaffri	b7290f01d8	Batching for hf_pipeline (#10795) ... The huggingface pipeline in langchain (used for locally hosted models) does not support batching. If you send in a batch of prompts, it just processes them serially using the base implementation of _generate: https://github.com/docugami/langchain/blob/master/libs/langchain/langchain/llms/base.py#L1004C2-L1004C29 This PR adds support for batching in this pipeline, so that GPUs can be fully saturated. I updated the accompanying notebook to show GPU batch inference. --------- Co-authored-by: Taqi Jaffri <tjaffri@docugami.com>	2023-09-25 18:23:11 +01:00
William FH	d4f790fd40	Fix imports in notebook (#9458)	2023-08-18 10:08:47 -07:00
Bagatur	c8c8635dc9	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00