|
|
|
@ -19,8 +19,29 @@
|
|
|
|
|
"First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n",
|
|
|
|
|
"\n",
|
|
|
|
|
"* [Download](https://ollama.ai/download)\n",
|
|
|
|
|
"* Fetch a model, e.g., `Llama-7b`: `ollama pull llama2`\n",
|
|
|
|
|
"* Run `ollama run llama2`\n",
|
|
|
|
|
"* Fetch a model via `ollama pull <model family>`\n",
|
|
|
|
|
"* e.g., for `Llama-7b`: `ollama pull llama2` (see full list [here](https://github.com/jmorganca/ollama))\n",
|
|
|
|
|
"* This will download the most basic version of the model typically (e.g., smallest # parameters and `q4_0`)\n",
|
|
|
|
|
"* On Mac, it will download to \n",
|
|
|
|
|
"\n",
|
|
|
|
|
"`~/.ollama/models/manifests/registry.ollama.ai/library/<model family>/latest`\n",
|
|
|
|
|
"\n",
|
|
|
|
|
"* And we specify a particular version, e.g., for `ollama pull vicuna:13b-v1.5-16k-q4_0`\n",
|
|
|
|
|
"* The file is here with the model version in place of `latest`\n",
|
|
|
|
|
"\n",
|
|
|
|
|
"`~/.ollama/models/manifests/registry.ollama.ai/library/vicuna/13b-v1.5-16k-q4_0`\n",
|
|
|
|
|
"\n",
|
|
|
|
|
"You can easily access models in a few ways:\n",
|
|
|
|
|
"\n",
|
|
|
|
|
"1/ if the app is running:\n",
|
|
|
|
|
"* All of your local models are automatically served on `localhost:11434`\n",
|
|
|
|
|
"* Select your model when setting `llm = Ollama(..., model=\"<model family>:<version>\")`\n",
|
|
|
|
|
"* If you set `llm = Ollama(..., model=\"<model family\")` withoout a version it will simply look for `latest`\n",
|
|
|
|
|
"\n",
|
|
|
|
|
"2/ if building from source or just running the binary: \n",
|
|
|
|
|
"* Then you must run `ollama serve`\n",
|
|
|
|
|
"* All of your local models are automatically served on `localhost:11434`\n",
|
|
|
|
|
"* Then, select as shown above\n",
|
|
|
|
|
"\n",
|
|
|
|
|
"\n",
|
|
|
|
|
"## Usage\n",
|
|
|
|
|