From be367019d6b32fd5ce0c01b9420d4c080775dd9d Mon Sep 17 00:00:00 2001 From: Andriy Mulyar Date: Wed, 10 May 2023 12:05:42 -0400 Subject: [PATCH] Update README.md --- README.md | 272 +++++------------------------------------------------- 1 file changed, 25 insertions(+), 247 deletions(-) diff --git a/README.md b/README.md index 078c6203..6862a562 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,19 @@

GPT4All

Demo, data, and code to train open-source assistant-style large language model based on GPT-J and LLaMa

+ +

+GPT4All Website +

+ +

+Discord +

+ + +

+:green_book: Technical Report 3: GPT4All Snoozy and Groovy +

+

:green_book: Technical Report 2: GPT4All-J

@@ -16,21 +30,16 @@ :computer: Official Typescript Bindings

-

-:speech_balloon: Official Web Chat Interface -

-

:speech_balloon: Official Chat Interface

-🦜️🔗 Official Langchain Backend +:speech_balloon: Official Web Chat Interface

-

-Discord +🦜️🔗 Official Langchain Backend

@@ -42,14 +51,20 @@ GPT4All is made possible by our compute partner +GPT4All Website +

+ +Direct Installer Links: [Mac/OSX](https://gpt4all.io/installers/gpt4all-installer-darwin.dmg) @@ -65,16 +80,9 @@ If you have older hardware that only supports avx and not avx2 you can use these [Ubuntu - avx-only](https://gpt4all.io/installers/gpt4all-installer-linux-avx-only.run) -These files are not yet cert signed by Windows/Apple so you will see security warnings on initial installation. We did not want to delay release while waiting for their process to complete. Find the most up-to-date information on the [GPT4All Website](https://gpt4all.io/) -### Raw Model -[ggml Model Download Link](https://gpt4all.io/models/ggml-gpt4all-j.bin) - -Note this model is only compatible with the C++ bindings found [here](https://github.com/nomic-ai/gpt4all-chat). It will not work with any existing llama.cpp bindings as we had to do a large fork of llama.cpp. GPT4All will support the ecosystem around this new C++ backend going forward. - -Python bindings are imminent and will be integrated into this [repository](https://github.com/nomic-ai/pyllamacpp). Stay tuned on the [GPT4All discord](https://discord.gg/mGZE39AS3e) for updates. ## Training GPT4All-J @@ -109,236 +117,6 @@ model = AutoModelForCausalLM.from_pretrained("nomic-ai/gpt4all-j-prompt-generati ```bash accelerate launch --dynamo_backend=inductor --num_processes=8 --num_machines=1 --machine_rank=0 --deepspeed_multinode_launcher standard --mixed_precision=bf16 --use_deepspeed --deepspeed_config_file=configs/deepspeed/ds_config_gptj.json train.py --config configs/train/finetune_gptj.yaml ``` - - -# Original GPT4All Model (based on GPL Licensed LLaMa) - - - -![gpt4all-lora-demo](https://user-images.githubusercontent.com/13879686/228352356-de66ca7a-df70-474e-b929-2e3656165051.gif) - -Run on M1 Mac (not sped up!) - -# Try it yourself - -Here's how to get started with the CPU quantized GPT4All model checkpoint: - -1. Download the `gpt4all-lora-quantized.bin` file from [Direct Link](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized.bin) or [[Torrent-Magnet]](https://tinyurl.com/gpt4all-lora-quantized). -2. Clone this repository, navigate to `chat`, and place the downloaded file there. -3. Run the appropriate command for your OS: - - M1 Mac/OSX: `cd chat;./gpt4all-lora-quantized-OSX-m1` - - Linux: `cd chat;./gpt4all-lora-quantized-linux-x86` - - Windows (PowerShell): `cd chat;./gpt4all-lora-quantized-win64.exe` - - Intel Mac/OSX: `cd chat;./gpt4all-lora-quantized-OSX-intel` - -For custom hardware compilation, see our [llama.cpp](https://github.com/zanussbaum/gpt4all.cpp) fork. - ------------ -Find all compatible models in the GPT4All Ecosystem section. - -[Secret Unfiltered Checkpoint](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-unfiltered-quantized.bin) - [[Torrent]](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-unfiltered-quantized.bin.torrent) - -This model had all refusal to answer responses removed from training. Try it with: -- M1 Mac/OSX: `cd chat;./gpt4all-lora-quantized-OSX-m1 -m gpt4all-lora-unfiltered-quantized.bin` -- Linux: `cd chat;./gpt4all-lora-quantized-linux-x86 -m gpt4all-lora-unfiltered-quantized.bin` -- Windows (PowerShell): `cd chat;./gpt4all-lora-quantized-win64.exe -m gpt4all-lora-unfiltered-quantized.bin` -- Intel Mac/OSX: `cd chat;./gpt4all-lora-quantized-OSX-intel -m gpt4all-lora-unfiltered-quantized.bin` ------------ -Note: the full model on GPU (16GB of RAM required) performs much better in our qualitative evaluations. - -# Python Client -## CPU Interface -To run GPT4All in python, see the new [official Python bindings](https://github.com/nomic-ai/pyllamacpp). - -The old bindings are still available but now deprecated. They will not work in a notebook environment. -To get running using the python client with the CPU interface, first install the [nomic client](https://github.com/nomic-ai/nomic) using `pip install nomic` -Then, you can use the following script to interact with GPT4All: -``` -from nomic.gpt4all import GPT4All -m = GPT4All() -m.open() -m.prompt('write me a story about a lonely computer') -``` - -## GPU Interface -There are two ways to get up and running with this model on GPU. -The setup here is slightly more involved than the CPU model. -1. clone the nomic client [repo](https://github.com/nomic-ai/nomic) and run `pip install .[GPT4All]` in the home dir. -2. run `pip install nomic` and install the additional deps from the wheels built [here](https://github.com/nomic-ai/nomic/tree/main/bin) - -Once this is done, you can run the model on GPU with a script like the following: -``` -from nomic.gpt4all import GPT4AllGPU -m = GPT4AllGPU(LLAMA_PATH) -config = {'num_beams': 2, - 'min_new_tokens': 10, - 'max_length': 100, - 'repetition_penalty': 2.0} -out = m.generate('write me a story about a lonely computer', config) -print(out) -``` -Where LLAMA_PATH is the path to a Huggingface Automodel compliant LLAMA model. -Nomic is unable to distribute this file at this time. -We are working on a GPT4All that does not have this limitation right now. - -You can pass any of the [huggingface generation config params](https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig) in the config. - -# GPT4All Compatibility Ecosystem -Edge models in the GPT4All Ecosystem. Please PR as the [community grows](https://huggingface.co/models?sort=modified&search=4bit). -Feel free to convert this to a more structured table. - -- [gpt4all](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized.bin) [[MD5 Signature](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized.bin.md5)] - - [gpt4all-ggml-converted](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized-ggml.bin) [[MD5 Signature](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized-ggml.bin.md5)] -- [gpt4all-unfiltered](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-unfiltered-quantized.bin) [[MD5 Signature](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-unfiltered-quantized.bin.md5)] -- [ggml-vicuna-7b-4bit](https://huggingface.co/eachadea/ggml-vicuna-7b-4bit) -- [vicuna-13b-GPTQ-4bit-128g](https://huggingface.co/anon8231489123/vicuna-13b-GPTQ-4bit-128g) -- [LLaMa-Storytelling-4Bit](https://huggingface.co/GamerUntouch/LLaMa-Storytelling-4Bit) -- [Alpaca Native 4bit](https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml/tree/main) - - -# Roadmap -## Short Term - - (Done) Train a GPT4All model based on GPTJ to alleviate llama distribution issues. - - (Done) Create improved CPU and GPU interfaces for this model. - - (Done) [Integrate llama.cpp bindings](https://github.com/nomic-ai/pyllamacpp) - - (Done) [Create a good conversational chat interface for the model.](https://github.com/nomic-ai/gpt4all-ui) - - (Done) [Allow users to opt in and submit their chats for subsequent training runs](https://github.com/nomic-ai/gpt4all-ui) - -## Medium Term - - (NOT STARTED) Integrate GPT4All with [Atlas](https://atlas.nomic.ai) to allow for document retrieval. - - BLOCKED by GPT4All based on GPTJ - - (Done) Integrate GPT4All with Langchain. - - (IN PROGRESS) Build easy custom training scripts to allow users to fine tune models. - -## Long Term - - (NOT STARTED) Allow anyone to curate training data for subsequent GPT4All releases using Atlas. - - (IN PROGRESS) Democratize AI. - -# Reproducibility - -Trained Model Weights: -- gpt4all-lora (four full epochs of training): https://huggingface.co/nomic-ai/gpt4all-lora -- gpt4all-lora-epoch-2 (three full epochs of training) https://huggingface.co/nomic-ai/gpt4all-lora-epoch-2 -- gpt4all-j (one full epoch of training) (https://huggingface.co/nomic-ai/gpt4all-j) -- gpt4all-j-lora (one full epoch of training) (https://huggingface.co/nomic-ai/gpt4all-j-lora) - -Raw Data: -- [Training Data Without P3](https://huggingface.co/datasets/nomic-ai/gpt4all_prompt_generations) - - Explorer: https://atlas.nomic.ai/map/gpt4all_data_clean_without_p3 -- [Full Dataset with P3](https://huggingface.co/datasets/nomic-ai/gpt4all_prompt_generations_with_p3) - - Explorer: https://atlas.nomic.ai/map/gpt4all_data_clean -- [GPT4All-J Dataset](https://huggingface.co/datasets/nomic-ai/gpt4all-j-prompt-generations) - - Explorer Indexed on Prompts: https://atlas.nomic.ai/map/gpt4all-j-prompts-curated - - Exporer Indexed on Responses: https://atlas.nomic.ai/map/gpt4all-j-response-curated - -We are not distributing a LLaMa 7B checkpoint. - -You can reproduce our trained model by doing the following: - -## Setup - -Clone the repo - -``` -git clone --recurse-submodules https://github.com/nomic-ai/gpt4all.git -git submodule update --init -``` - -Setup the environment - -``` -python -m pip install -r requirements.txt - -cd ../peft -pip install -e . -``` - -## Training - -```bash -accelerate launch --dynamo_backend=inductor --num_processes=8 --num_machines=1 --machine_rank=0 --deepspeed_multinode_launcher standard --mixed_precision=bf16 --use_deepspeed --deepspeed_config_file=configs/deepspeed/ds_config.json train.py --config configs/train/finetune-7b.yaml -``` - -## Generate - -```bash -python generate.py --config configs/generate/generate.yaml --prompt "Write a script to reverse a string in Python" -``` - -## Need Help? - -Join the Discord and ask for help in `#gpt4all-help` - -# Sample Generations - -### Provide instructions for the given exercise. Leg Raises ->Leg Raises ->1. Stand with your feet shoulder-width apart and your knees slightly bent. ->2. Raise your arms straight out in front of you. ->3. Slowly bend your knees and raise your heels off the ground. ->4. Hold this position for a count of 3, then slowly lower your heels back to the ground. ->5. Repeat the exercise 10 times. ->6. Do not let your knees go beyond your toes. ->7. Keep your back straight and your abdominal muscles engaged. ->8. Do not let your hips sag or your back arch. ->9. Keep your arms straight and your elbows locked. ->10. Do not let your head drop or your shoulders rise. ->11. Breathe normally throughout the exercise. ->12. Rest for 30 seconds before repeating the exercise. ->13. Gradually increase the number of repetitions as you become stronger. - - -### A color description has been provided. Find the CSS code associated with that color. A light red color with a medium light shade of pink - ->The CSS code associated with the color provided is: #FF6347 - - -### Come up with an interesting idea for a new movie plot. Your plot should be described with a title and a summary. - -> 1. The title of your movie plot should be "The Last Stand". -> 2. The summary of your movie plot should be: A group of mercenaries are hired to protect a small town from a dangerous criminal who is on the run. However, the town's sheriff is not willing to let them take over and decides to take matters into his own hands. - -### Reverse a string in python. -> In Python, you can reverse a list or tuple by using the reversed() function on it. Here's an example of how to use this method with strings: -> ```python -> my_string = "Hello World" # Define your original string here -> reversed_str = my_string[::-1] # Use a slice and negative index for reverse order -> print(reversed_str) # Output will be reversed version of the inputted string. - -### List 10 dogs. -> 1. Labrador Retriever -> 2. Golden Retriever -> 3. Beagle -> 4. Poodle -> 5. German Shepherd -> 6. Bulldog -> 7. Rottweiler -> 8. Boxer -> 9. Shih Tzu -> 10. Pug - -### Write me a poem about the fall of Julius Caesar into a caesar salad in iambic pentameter. - -> The fall of Julius Caesar into a caesar salad
-> In iambic pentameter,
-> The mighty Roman emperor
-> Fell into a salad of lettuce and croutons,
-> His empire crumbling around him,
-> As he was devoured by the hungry diners.
-> The once mighty emperor
-> Was now just a salad topping,
-> His legacy reduced to a mere garnish,
-> As he was consumed by the masses.
-> The fall of Julius Caesar into a caesar salad,
-> In iambic pentameter,
-> Was a tragic end to a great man,
-> Who had once ruled the world with an iron fist.
-> But now, he was just a salad topping,
-> His legacy reduced to a mere garnish,
-> As he was consumed by the masses. - -### What is a three word topic describing the following keywords: baseball, football, soccer: -> Sports, athletics, games ## Citation