mirror of
https://github.com/hwchase17/langchain
synced 2024-10-29 17:07:25 +00:00
b3988621c5
# Add C Transformers for GGML Models I created Python bindings for the GGML models: https://github.com/marella/ctransformers Currently it supports GPT-2, GPT-J, GPT-NeoX, LLaMA, MPT, etc. See [Supported Models](https://github.com/marella/ctransformers#supported-models). It provides a unified interface for all models: ```python from langchain.llms import CTransformers llm = CTransformers(model='/path/to/ggml-gpt-2.bin', model_type='gpt2') print(llm('AI is going to')) ``` It can be used with models hosted on the Hugging Face Hub: ```py llm = CTransformers(model='marella/gpt-2-ggml') ``` It supports streaming: ```py from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler llm = CTransformers(model='marella/gpt-2-ggml', callbacks=[StreamingStdOutCallbackHandler()]) ``` Please see [README](https://github.com/marella/ctransformers#readme) for more details. --------- Co-authored-by: Dev 2049 <dev.dev2049@gmail.com> |
||
---|---|---|
.. | ||
agent | ||
cache | ||
callbacks | ||
chains | ||
chat_models | ||
client | ||
document_loaders | ||
embeddings | ||
examples | ||
llms | ||
memory | ||
prompts | ||
retrievers | ||
utilities | ||
vectorstores | ||
__init__.py | ||
.env.example | ||
conftest.py | ||
test_document_transformers.py | ||
test_nlp_text_splitters.py | ||
test_pdf_pagesplitter.py | ||
test_schema.py | ||
test_text_splitter.py |