docs: add llamafile info to 'Local LLMs' guides (#18049)

- **Description:** add information about
[llamafile](https://github.com/Mozilla-Ocho/llamafile) (setup, example
usage) to ['Run LLMs
locally'](https://python.langchain.com/docs/guides/local_llms) and
['Using local models for Q&A with
RAG'](https://python.langchain.com/docs/use_cases/question_answering/local_retrieval_qa)
guides.
- **Issue:** N/A
- **Dependencies:** N/A

docs/docs/guides/local_llms.ipynb

@@ -9,7 +9,7 @@
     "\n",
     "## Use case\n",
     "\n",
-    "The popularity of projects like [PrivateGPT](https://github.com/imartinez/privateGPT), [llama.cpp](https://github.com/ggerganov/llama.cpp), and [GPT4All](https://github.com/nomic-ai/gpt4all) underscore the demand to run LLMs locally (on your own device).\n",
+    "The popularity of projects like [PrivateGPT](https://github.com/imartinez/privateGPT), [llama.cpp](https://github.com/ggerganov/llama.cpp), [GPT4All](https://github.com/nomic-ai/gpt4all), and [llamafile](https://github.com/Mozilla-Ocho/llamafile) underscore the demand to run LLMs locally (on your own device).\n",
     "\n",
     "This has at least two important benefits:\n",
     "\n",
@@ -46,7 +46,8 @@
     "\n",
     "1. [`llama.cpp`](https://github.com/ggerganov/llama.cpp): C++ implementation of llama inference code with [weight optimization / quantization](https://finbarr.ca/how-is-llama-cpp-possible/)\n",
     "2. [`gpt4all`](https://docs.gpt4all.io/index.html): Optimized C backend for inference\n",
-    "3. [`Ollama`](https://ollama.ai/): Bundles model weights and environment into an app that runs on device and serves the LLM \n",
+    "3. [`Ollama`](https://ollama.ai/): Bundles model weights and environment into an app that runs on device and serves the LLM\n",
+    "4. [`llamafile`](https://github.com/Mozilla-Ocho/llamafile): Bundles model weights and everything needed to run the model in a single file, allowing you to run the LLM locally from this file without any additional installation steps\n",
     "\n",
     "In general, these frameworks will do a few things:\n",
     "\n",
@@ -157,7 +158,7 @@
     "\n",
     "### Running Apple silicon GPU\n",
     "\n",
-    "`Ollama` will automatically utilize the GPU on Apple devices.\n",
+    "`Ollama` and [`llamafile`](https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file#gpu-support) will automatically utilize the GPU on Apple devices.\n",
     " \n",
     "Other frameworks require the user to set up the environment to utilize the Apple GPU.\n",
     "\n",
@@ -191,7 +192,7 @@
     "\n",
     "There are various ways to gain access to quantized model weights.\n",
     "\n",
-    "1. [`HuggingFace`](https://huggingface.co/TheBloke) - Many quantized model are available for download and can be run with framework such as [`llama.cpp`](https://github.com/ggerganov/llama.cpp)\n",
+    "1. [`HuggingFace`](https://huggingface.co/TheBloke) - Many quantized model are available for download and can be run with framework such as [`llama.cpp`](https://github.com/ggerganov/llama.cpp). You can also download models in [`llamafile` format](https://huggingface.co/models?other=llamafile) from HuggingFace.\n",
     "2. [`gpt4all`](https://gpt4all.io/index.html) - The model explorer offers a leaderboard of metrics and associated quantized models available for download \n",
     "3. [`Ollama`](https://github.com/jmorganca/ollama) - Several models can be accessed directly via `pull`\n",
     "\n",
@@ -428,6 +429,62 @@
     "llm(\"The first man on the moon was ... Let's think step by step\")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "056854e2-5e4b-4a03-be7e-03192e5c4e1e",
+   "metadata": {},
+   "source": [
+    "### llamafile\n",
+    "\n",
+    "One of the simplest ways to run an LLM locally is using a [llamafile](https://github.com/Mozilla-Ocho/llamafile). All you need to do is:\n",
+    "\n",
+    "1) Download a llamafile from [HuggingFace](https://huggingface.co/models?other=llamafile)\n",
+    "2) Make the file executable\n",
+    "3) Run the file\n",
+    "\n",
+    "llamafiles bundle model weights and a [specially-compiled](https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file#technical-details) version of [`llama.cpp`](https://github.com/ggerganov/llama.cpp) into a single file that can run on most computers any additional dependencies. They also come with an embedded inference server that provides an [API](https://github.com/Mozilla-Ocho/llamafile/blob/main/llama.cpp/server/README.md#api-endpoints) for interacting with your model. \n",
+    "\n",
+    "Here's a simple bash script that shows all 3 setup steps:\n",
+    "\n",
+    "```bash\n",
+    "# Download a llamafile from HuggingFace\n",
+    "wget https://huggingface.co/jartine/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile\n",
+    "\n",
+    "# Make the file executable. On Windows, instead just rename the file to end in \".exe\".\n",
+    "chmod +x TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile\n",
+    "\n",
+    "# Start the model server. Listens at http://localhost:8080 by default.\n",
+    "./TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile --server --nobrowser\n",
+    "```\n",
+    "\n",
+    "After you run the above setup steps, you can use LangChain to interact with your model:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "002e655c-ba18-4db3-ac7b-f33e825d14b6",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"\\nFirstly, let's imagine the scene where Neil Armstrong stepped onto the moon. This happened in 1969. The first man on the moon was Neil Armstrong. We already know that.\\n2nd, let's take a step back. Neil Armstrong didn't have any special powers. He had to land his spacecraft safely on the moon without injuring anyone or causing any damage. If he failed to do this, he would have been killed along with all those people who were on board the spacecraft.\\n3rd, let's imagine that Neil Armstrong successfully landed his spacecraft on the moon and made it back to Earth safely. The next step was for him to be hailed as a hero by his people back home. It took years before Neil Armstrong became an American hero.\\n4th, let's take another step back. Let's imagine that Neil Armstrong wasn't hailed as a hero, and instead, he was just forgotten. This happened in the 1970s. Neil Armstrong wasn't recognized for his remarkable achievement on the moon until after he died.\\n5th, let's take another step back. Let's imagine that Neil Armstrong didn't die in the 1970s and instead, lived to be a hundred years old. This happened in 2036. In the year 2036, Neil Armstrong would have been a centenarian.\\nNow, let's think about the present. Neil Armstrong is still alive. He turned 95 years old on July 20th, 2018. If he were to die now, his achievement of becoming the first human being to set foot on the moon would remain an unforgettable moment in history.\\nI hope this helps you understand the significance and importance of Neil Armstrong's achievement on the moon!\""
+      ]
+     },
+     "execution_count": 1,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain_community.llms.llamafile import Llamafile\n",
+    "\n",
+    "llm = Llamafile()\n",
+    "\n",
+    "llm.invoke(\"The first man on the moon was ... Let's think step by step.\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "6b84e543",
@@ -611,7 +668,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.11.7"
   }
  },
  "nbformat": 4,

docs/docs/use_cases/question_answering/local_retrieval_qa.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Using local models\n",
     "\n",
-    "The popularity of projects like [PrivateGPT](https://github.com/imartinez/privateGPT), [llama.cpp](https://github.com/ggerganov/llama.cpp), and [GPT4All](https://github.com/nomic-ai/gpt4all) underscore the importance of running LLMs locally.\n",
+    "The popularity of projects like [PrivateGPT](https://github.com/imartinez/privateGPT), [llama.cpp](https://github.com/ggerganov/llama.cpp), [GPT4All](https://github.com/nomic-ai/gpt4all), and [llamafile](https://github.com/Mozilla-Ocho/llamafile) underscore the importance of running LLMs locally.\n",
     "\n",
     "LangChain has [integrations](https://integrations.langchain.com/) with many open-source LLMs that can be run locally.\n",
     "\n",
@@ -330,6 +330,62 @@
     ")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "e6d012e4-0eef-4734-a826-89ec74fe9f88",
+   "metadata": {},
+   "source": [
+    "### llamafile\n",
+    "\n",
+    "One of the simplest ways to run an LLM locally is using a [llamafile](https://github.com/Mozilla-Ocho/llamafile). All you need to do is:\n",
+    "\n",
+    "1) Download a llamafile from [HuggingFace](https://huggingface.co/models?other=llamafile)\n",
+    "2) Make the file executable\n",
+    "3) Run the file\n",
+    "\n",
+    "llamafiles bundle model weights and a [specially-compiled](https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file#technical-details) version of [`llama.cpp`](https://github.com/ggerganov/llama.cpp) into a single file that can run on most computers without any additional dependencies. They also come with an embedded inference server that provides an [API](https://github.com/Mozilla-Ocho/llamafile/blob/main/llama.cpp/server/README.md#api-endpoints) for interacting with your model. \n",
+    "\n",
+    "Here's a simple bash script that shows all 3 setup steps:\n",
+    "\n",
+    "```bash\n",
+    "# Download a llamafile from HuggingFace\n",
+    "wget https://huggingface.co/jartine/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile\n",
+    "\n",
+    "# Make the file executable. On Windows, instead just rename the file to end in \".exe\".\n",
+    "chmod +x TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile\n",
+    "\n",
+    "# Start the model server. Listens at http://localhost:8080 by default.\n",
+    "./TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile --server --nobrowser\n",
+    "```\n",
+    "\n",
+    "After you run the above setup steps, you can interact with the model via LangChain:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "735e45b6-9aff-463e-aae4-bbf8ac2b21c5",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'\\n-1 1/2 (8 oz. Pounds) ground beef, browned and cooked until no longer pink\\n-3 cups whole wheat spaghetti\\n-4 (10 oz) cans diced tomatoes with garlic and basil\\n-2 eggs, beaten\\n-1 cup grated parmesan cheese\\n-1/2 teaspoon salt\\n-1/4 teaspoon black pepper\\n-1 cup breadcrumbs (16 oz)\\n-2 tablespoons olive oil\\n\\nInstructions:\\n1. Cook spaghetti according to package directions. Drain and set aside.\\n2. In a large skillet, brown ground beef over medium heat until no longer pink. Drain any excess grease.\\n3. Stir in diced tomatoes with garlic and basil, and season with salt and pepper. Cook for 5 to 7 minutes or until sauce is heated through. Set aside.\\n4. In a large bowl, beat eggs with a fork or whisk until fluffy. Add cheese, salt, and black pepper. Set aside.\\n5. In another bowl, combine breadcrumbs and olive oil. Dip each spaghetti into the egg mixture and then coat in the breadcrumb mixture. Place on baking sheet lined with parchment paper to prevent sticking. Repeat until all spaghetti are coated.\\n6. Heat oven to 375 degrees. Bake for 18 to 20 minutes, or until lightly golden brown.\\n7. Serve hot with meatballs and sauce on the side. Enjoy!'"
+      ]
+     },
+     "execution_count": 1,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain_community.llms.llamafile import Llamafile\n",
+    "\n",
+    "llamafile = Llamafile()\n",
+    "\n",
+    "llamafile.invoke(\"Here is my grandmother's beloved recipe for spaghetti and meatballs:\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "d58838ae",
@@ -702,7 +758,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.11.7"
   }
  },
  "nbformat": 4,