Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-10-29 17:07:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
89ef440c14
langchain/docs/extras/integrations/llms
History
Taqi Jaffri b7290f01d8
Batching for hf_pipeline (#10795) 
The huggingface pipeline in langchain (used for locally hosted models)
does not support batching. If you send in a batch of prompts, it just
processes them serially using the base implementation of _generate:
https://github.com/docugami/langchain/blob/master/libs/langchain/langchain/llms/base.py#L1004C2-L1004C29

This PR adds support for batching in this pipeline, so that GPUs can be
fully saturated. I updated the accompanying notebook to show GPU batch
inference.

---------

Co-authored-by: Taqi Jaffri <tjaffri@docugami.com>
2023-09-25 18:23:11 +01:00
..
ai21.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
aleph_alpha.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
amazon_api_gateway.ipynb 📖 docs: fixed integration/llms navbar (#9277) 2023-09-01 15:30:37 -07:00
anyscale.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
azure_ml.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
azure_openai.ipynb 📖 docs: fixed integration/llms navbar (#9277) 2023-09-01 15:30:37 -07:00
baidu_qianfan_endpoint.ipynb fix: chat_models Qianfan not compatiable with SystemMessage (#10642) 2023-09-19 22:35:51 -07:00
banana.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
baseten.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
beam.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
bedrock.ipynb Amazon Bedrock Support Streaming (#10393) 2023-09-20 11:55:38 -07:00
bittensor.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
cerebriumai.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
chatglm.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
clarifai.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
cohere.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
ctransformers.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
ctranslate2.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
databricks.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
deepinfra.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
deepsparse.ipynb Adds DeepSparse as an LLM (#9184) 2023-08-13 22:35:58 -07:00
edenai.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
fireworks.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
forefrontai.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
google_vertex_ai_palm.ipynb add vertex prod features (#10910) 2023-09-22 01:44:09 -07:00
gooseai.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
gpt4all.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
gradient.ipynb gradient.ai LLM intregration (#10800) 2023-09-21 07:29:16 -07:00
huggingface_hub.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
huggingface_pipelines.ipynb Batching for hf_pipeline (#10795) 2023-09-25 18:23:11 +01:00
huggingface_textgen_inference.ipynb Fix typo in huggingface_textgen_inference.ipynb (#9313) 2023-08-16 16:22:21 -07:00
index.mdx llm feat table revision (#10947) 2023-09-22 10:29:12 -07:00
javelin.ipynb Add Javelin integration (#10275) 2023-09-20 16:36:39 -07:00
jsonformer_experimental.ipynb Wfh/ref links (#8454) 2023-07-29 08:44:32 -07:00
koboldai.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
llamacpp.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
llm_caching.ipynb added Cassandra caches to the llm_caching notebook doc (#10889) 2023-09-21 08:52:52 -07:00
manifest.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
minimax.ipynb Add Minimax chat model (#10776) 2023-09-19 20:43:49 -07:00
modal.ipynb Add Minimax chat model (#10776) 2023-09-19 20:43:49 -07:00
mosaicml.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
nlpcloud.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
octoai.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
ollama.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
opaqueprompts.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
openai.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
openllm.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
openlm.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
petals.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
pipelineai.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
predibase.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
predictionguard.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
promptlayer_openai.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
rellm_experimental.ipynb Wfh/ref links (#8454) 2023-07-29 08:44:32 -07:00
replicate.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
runhouse.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
sagemaker.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
stochasticai.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
symblai_nebula.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
textgen.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
titan_takeoff.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
tongyi.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
vllm.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
writer.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00
xinference.ipynb Harrison/stop importing from init (#10690) 2023-09-16 17:22:48 -07:00