Update Ollama docs (#9220)

Based on discussion w/ team.
pull/9227/head
Lance Martin 1 year ago committed by GitHub
parent 3f601b5809
commit 17ae2998e7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -19,8 +19,29 @@
"First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n",
"\n",
"* [Download](https://ollama.ai/download)\n",
"* Fetch a model, e.g., `Llama-7b`: `ollama pull llama2`\n",
"* Run `ollama run llama2`\n",
"* Fetch a model via `ollama pull <model family>`\n",
"* e.g., for `Llama-7b`: `ollama pull llama2` (see full list [here](https://github.com/jmorganca/ollama))\n",
"* This will download the most basic version of the model typically (e.g., smallest # parameters and `q4_0`)\n",
"* On Mac, it will download to \n",
"\n",
"`~/.ollama/models/manifests/registry.ollama.ai/library/<model family>/latest`\n",
"\n",
"* And we specify a particular version, e.g., for `ollama pull vicuna:13b-v1.5-16k-q4_0`\n",
"* The file is here with the model version in place of `latest`\n",
"\n",
"`~/.ollama/models/manifests/registry.ollama.ai/library/vicuna/13b-v1.5-16k-q4_0`\n",
"\n",
"You can easily access models in a few ways:\n",
"\n",
"1/ if the app is running:\n",
"* All of your local models are automatically served on `localhost:11434`\n",
"* Select your model when setting `llm = Ollama(..., model=\"<model family>:<version>\")`\n",
"* If you set `llm = Ollama(..., model=\"<model family\")` withoout a version it will simply look for `latest`\n",
"\n",
"2/ if building from source or just running the binary: \n",
"* Then you must run `ollama serve`\n",
"* All of your local models are automatically served on `localhost:11434`\n",
"* Then, select as shown above\n",
"\n",
"\n",
"## Usage\n",

Loading…
Cancel
Save