You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
langchain/docs/extras
Taqi Jaffri b7290f01d8
Batching for hf_pipeline (#10795)
The huggingface pipeline in langchain (used for locally hosted models)
does not support batching. If you send in a batch of prompts, it just
processes them serially using the base implementation of _generate:
https://github.com/docugami/langchain/blob/master/libs/langchain/langchain/llms/base.py#L1004C2-L1004C29

This PR adds support for batching in this pipeline, so that GPUs can be
fully saturated. I updated the accompanying notebook to show GPU batch
inference.

---------

Co-authored-by: Taqi Jaffri <tjaffri@docugami.com>
12 months ago
..
_templates
additional_resources
expression_language add lcel how to index (#10850) 1 year ago
guides
integrations Batching for hf_pipeline (#10795) 12 months ago
modules Harrison/agents structured (#10911) 1 year ago
use_cases beef up agent docs (#10866) 1 year ago