mirror of
https://github.com/hwchase17/langchain
synced 2024-11-06 03:20:49 +00:00
0bc4a9b3fc
* **Description:** Adds a simple LLM implementation for interacting with [llamafile](https://github.com/Mozilla-Ocho/llamafile)-based models. * **Dependencies:** N/A * **Issue:** N/A **Detail** [llamafile](https://github.com/Mozilla-Ocho/llamafile) lets you run LLMs locally from a single file on most computers without installing any dependencies. To use the llamafile LLM implementation, the user needs to: 1. Download a llamafile e.g. https://huggingface.co/jartine/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile?download=true 2. Make the file executable. 3. Run the llamafile in 'server mode'. (All llamafiles come packaged with a lightweight server; by default, the server listens at `http://localhost:8080`.) ```bash wget https://url/of/model.llamafile chmod +x model.llamafile ./model.llamafile --server --nobrowser ``` Now, the user can invoke the LLM via the LangChain client: ```python from langchain_community.llms.llamafile import Llamafile llm = Llamafile() llm.invoke("Tell me a joke.") ``` |
||
|---|---|---|
| .. | ||
| agent_toolkits | ||
| callbacks | ||
| chat_loaders | ||
| chat_message_histories | ||
| chat_models | ||
| docstore | ||
| document_loaders | ||
| document_transformers | ||
| embeddings | ||
| examples | ||
| graphs | ||
| indexes | ||
| llms | ||
| retrievers | ||
| storage | ||
| tools | ||
| utilities | ||
| utils | ||
| vectorstores | ||
| __init__.py | ||
| conftest.py | ||
| test_dependencies.py | ||
| test_imports.py | ||
| test_sql_database_schema.py | ||
| test_sql_database.py | ||
| test_sqlalchemy.py | ||