Fix llama2 provider, Update TOC in readme (#1325)

pull/1333/head
H Lohaus 7 months ago committed by GitHub
parent 54602aa1b3
commit c3ccc4e819
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -22,36 +22,37 @@ docker pull hlohaus789/g4f
## 📚 Table of Contents
- [🆕 What's New](#-whats-new)
- [🆕 What's New](#-what-s-new)
- [📚 Table of Contents](#-table-of-contents)
- [🛠️ Getting Started](#-getting-started)
- [Prerequisites:](#prerequisites)
- [Setting up the project:](#setting-up-the-project)
- [🛠️ Getting Started](#-getting-started)
+ [Docker container](#docker-container)
- [Quick start](#quick-start)
+ [Use python package](#use-python-package)
- [Prerequisites](#prerequisites)
- [Install using pypi](#install-using-pypi)
- [or](#or)
- [Setting up with Docker:](#setting-up-with-docker)
+ [Docker for Developers](#docker-for-developers)
- [💡 Usage](#-usage)
- [The `g4f` Package](#the-g4f-package)
- [ChatCompletion](#chatcompletion)
* [The `g4f` Package](#the-g4f-package)
+ [ChatCompletion](#chatcompletion)
- [Completion](#completion)
- [Providers](#providers)
- [Cookies Required](#cookies-required)
- [Using Browser](#using-browser)
- [Async Support](#async-support)
- [Proxy and Timeout Support](#proxy-and-timeout-support)
- [Interference openai-proxy API (Use with openai python package)](#interference-openai-proxy-api-use-with-openai-python-package)
- [Run interference API from PyPi package](#run-interference-api-from-pypi-package)
- [Run interference API from repo](#run-interference-api-from-repo)
* [Interference openai-proxy API](#interference-openai-proxy-api-use-with-openai-python-package-)
+ [Run interference API from PyPi package](#run-interference-api-from-pypi-package)
+ [Run interference API from repo](#run-interference-api-from-repo)
- [🚀 Providers and Models](#-providers-and-models)
- [GPT-4](#gpt-4)
- [GPT-3.5](#gpt-35)
- [Other](#other)
- [Models](#models)
* [GPT-4](#gpt-4)
* [GPT-3.5](#gpt-35)
* [Other](#other)
* [Models](#models)
- [🔗 Related GPT4Free Projects](#-related-gpt4free-projects)
- [🤝 Contribute](#-contribute)
- [Create Provider with AI Tool](#create-provider-with-ai-tool)
- [Create Provider](#create-provider)
+ [Create Provider with AI Tool](#create-provider-with-ai-tool)
+ [Create Provider](#create-provider)
- [🙌 Contributors](#-contributors)
- [©️ Copyright](#-copyright)
- [©️ Copyright](#-copyright)
- [⭐ Star History](#-star-history)
- [📄 License](#-license)
@ -70,7 +71,7 @@ docker run -p 8080:80 -p 1337:1337 -p 7900:7900 --shm-size="2g" hlohaus789/g4f:l
```
3. Open the included client on: [http://localhost:8080/chat/](http://localhost:8080/chat/)
or set the api base in your client to: [http://localhost:1337/v1](http://localhost:1337/v1)
4. (Optional) If you need to log in to a provider, you can open the desktop in the container here: http://localhost:7900/?autoconnect=1&resize=scale&password=secret.
4. (Optional) If you need to log in to a provider, you can view the desktop from the container here: http://localhost:7900/?autoconnect=1&resize=scale&password=secret.
#### Use python package

@ -7,8 +7,8 @@ services:
build:
context: .
dockerfile: docker/Dockerfile
# volumes:
# - .:/app
volumes:
- .:/app
ports:
- '8080:80'
- '1337:1337'

@ -6,10 +6,9 @@ from ..typing import AsyncResult, Messages
from .base_provider import AsyncGeneratorProvider
models = {
"meta-llama/Llama-2-7b-chat-hf": {"name": "Llama 2 7B", "version": "d24902e3fa9b698cc208b5e63136c4e26e828659a9f09827ca6ec5bb83014381", "shortened":"7B"},
"meta-llama/Llama-2-13b-chat-hf": {"name": "Llama 2 13B", "version": "9dff94b1bed5af738655d4a7cbcdcde2bd503aa85c94334fe1f42af7f3dd5ee3", "shortened":"13B"},
"meta-llama/Llama-2-70b-chat-hf": {"name": "Llama 2 70B", "version": "2796ee9483c3fd7aa2e171d38f4ca12251a30609463dcfd4cd76703f22e96cdf", "shortened":"70B"},
"Llava": {"name": "Llava 13B", "version": "6bc1c7bb0d2a34e413301fee8f7cc728d2d4e75bfab186aa995f63292bda92fc", "shortened":"Llava"}
"meta-llama/Llama-2-7b-chat-hf": "meta/llama-2-7b-chat",
"meta-llama/Llama-2-13b-chat-hf": "meta/llama-2-13b-chat",
"meta-llama/Llama-2-70b-chat-hf": "meta/llama-2-70b-chat",
}
class Llama2(AsyncGeneratorProvider):
@ -26,10 +25,9 @@ class Llama2(AsyncGeneratorProvider):
**kwargs
) -> AsyncResult:
if not model:
model = "meta-llama/Llama-2-70b-chat-hf"
elif model not in models:
raise ValueError(f"Model are not supported: {model}")
version = models[model]["version"]
model = "meta/llama-2-70b-chat"
elif model in models:
model = models[model]
headers = {
"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/118.0",
"Accept": "*/*",
@ -50,7 +48,7 @@ class Llama2(AsyncGeneratorProvider):
prompt = format_prompt(messages)
data = {
"prompt": prompt,
"version": version,
"model": model,
"systemPrompt": kwargs.get("system_message", "You are a helpful assistant."),
"temperature": kwargs.get("temperature", 0.75),
"topP": kwargs.get("top_p", 0.9),

Loading…
Cancel
Save