8.0 KiB
GPT4All CLI
The GPT4All command-line interface (CLI) is a Python script which is built on top of the Python bindings (repository) and the typer package. The source code, README, and local build instructions can be found here.
Installation
The Short Version
The CLI is a Python script called app.py. If you're already familiar with Python best practices, the short version is to download app.py into a folder of your choice, install the two required dependencies with some variant of:
pip install gpt4all typer
Then run it with a variant of:
python app.py repl
In case you're wondering, REPL is an acronym for read-eval-print loop.
Recommendations & The Long Version
Especially if you have several applications/libraries which depend on Python, to avoid descending into dependency hell at some point, you should:
- Consider to always install into some kind of virtual environment.
- On a Unix-like system, don't use
sudo
for anything other than packages provided by the system package manager, i.e. never withpip
.
There are several ways and tools available to do this, so below are descriptions on how to install with a virtual environment (recommended) or a user installation on all three main platforms.
Different platforms can have slightly different ways to start the Python interpreter itself.
Note: Typer has an optional dependency for more fanciful output. If you want that, replace typer
with typer[all]
in the pip-install instructions below.
Virtual Environment Installation
You can name your virtual environment folder for the CLI whatever you like. In the following,
gpt4all-cli
is used throughout.
macOS
There are at least three ways to have a Python installation on macOS, and possibly not all of them provide a full installation of Python and its tools. When in doubt, try the following:
python3 -m venv --help
python3 -m pip --help
Both should print the help for the venv
and pip
commands, respectively. If they don't, consult
the documentation of your Python installation on how to enable them, or download a separate Python
variant, for example try an unified installer package from python.org.
Once ready, do:
python3 -m venv gpt4all-cli
. gpt4all-cli/bin/activate
python3 -m pip install gpt4all typer
Windows
Download the official installer from python.org if Python isn't already present on your system.
A Windows installation should already provide all the components for a virtual environment. Run:
py -3 -m venv gpt4all-cli
gpt4all-cli\Scripts\activate
py -m pip install gpt4all typer
Linux
On Linux, a Python installation is often split into several packages and not all are necessarily installed by default. For example, on Debian/Ubuntu and derived distros, you will want to ensure their presence with the following:
sudo apt-get install python3-venv python3-pip
The next steps are similar to the other platforms:
python3 -m venv gpt4all-cli
. gpt4all-cli/bin/activate
python3 -m pip install gpt4all typer
On other distros, the situation might be different. Especially the package names can vary a lot. You'll have to look it up in the documentation, software directory, or package search.
User Installation
macOS
There are at least three ways to have a Python installation on macOS, and possibly not all of them provide a full installation of Python and its tools. When in doubt, try the following:
python3 -m pip --help
That should print the help for the pip
command. If it doesn't, consult the documentation of your
Python installation on how to enable them, or download a separate Python variant, for example try an
unified installer package from python.org.
Once ready, do:
python3 -m pip install --user --upgrade gpt4all typer
Windows
Download the official installer from python.org if Python isn't already present on your system. It includes all the necessary components. Run:
py -3 -m pip install --user --upgrade gpt4all typer
Linux
On Linux, a Python installation is often split into several packages and not all are necessarily installed by default. For example, on Debian/Ubuntu and derived distros, you will want to ensure their presence with the following:
sudo apt-get install python3-pip
The next steps are similar to the other platforms:
python3 -m pip install --user --upgrade gpt4all typer
On other distros, the situation might be different. Especially the package names can vary a lot. You'll have to look it up in the documentation, software directory, or package search.
Running the CLI
The CLI is a self-contained script called app.py. As such, you can download and save it anywhere you like, as long as the Python interpreter has access to the mentioned dependencies.
Note: different platforms can have slightly different ways to start Python. Whereas below the
interpreter command is written as python
you typically want to type instead:
- On Unix-like systems:
python3
- On Windows:
py -3
The simplest way to start the CLI is:
python app.py repl
This automatically selects the groovy model and downloads it into the .cache/gpt4all/
folder
of your home directory, if not already present.
If you want to use a different model, you can do so with the -m
/--model
parameter. If only a
model file name is provided, it will again check in .cache/gpt4all/
and might start downloading.
If instead given a path to an existing model, the command could for example look like this:
python app.py repl --model /home/user/my-gpt4all-models/gpt4all-13b-snoozy-q4_0.gguf
When you're done and want to end a session, simply type /exit
.
To get help and information on all the available commands and options on the command-line, run:
python app.py --help
And while inside the running REPL, write /help
.
Note that if you've installed the required packages into a virtual environment, you don't need
to activate that every time you want to run the CLI. Instead, you can just start it with the Python
interpreter in the folder gpt4all-cli/bin/
(Unix-like) or gpt4all-cli/Script/
(Windows).
That also makes it easy to set an alias e.g. in Bash or PowerShell:
- Bash:
alias gpt4all="'/full/path/to/gpt4all-cli/bin/python' '/full/path/to/app.py' repl"
- PowerShell:
Function GPT4All-Venv-CLI {"C:\full\path\to\gpt4all-cli\Scripts\python.exe" "C:\full\path\to\app.py" repl} Set-Alias -Name gpt4all -Value GPT4All-Venv-CLI
Don't forget to save these in the start-up file of your shell.
Finally, if on Windows you see a box instead of an arrow ⇢
as the prompt character, you should
change the console font to one which offers better Unicode support.