gpt4all/gpt4all-api/docker-compose.yaml

version: "3.8"

services:
  gpt4all_api:
    image: gpt4all_api
    container_name: gpt4all_api
    restart: always #restart on error (usually code compilation from save during bad state)
    ports:
      - "4891:4891"
    environment:
      - APP_ENVIRONMENT=dev
      - WEB_CONCURRENCY=2
      - LOGLEVEL=debug
      - PORT=4891
      - model=ggml-mpt-7b-chat.bin
      - inference_mode=cpu
    volumes:
      - './gpt4all_api/app:/app'
    command: ["/start-reload.sh"]
GPU Inference Server (#1112) * feat: local inference server * fix: source to use bash + vars * chore: isort and black * fix: make file + inference mode * chore: logging * refactor: remove old links * fix: add new env vars * feat: hf inference server * refactor: remove old links * test: batch and single response * chore: black + isort * separate gpu and cpu dockerfiles * moved gpu to separate dockerfile * Fixed test endpoints * Edits to API. server won't start due to failed instantiation error * Method signature * fix: gpu_infer * tests: fix tests --------- Co-authored-by: Andriy Mulyar <andriy.mulyar@gmail.com> 2023-07-21 19:13:29 +00:00			`version: "3.8"`
GPT4All API Scaffolding. Matches OpenAI OpenAPI spec for chats and completions (#839) * GPT4All API Scaffolding. Matches OpenAI OpenAI spec for engines, chats and completions * Edits for docker building * FastAPI app builds and pydantic models are accurate * Added groovy download into dockerfile * improved dockerfile * Chat completions endpoint edits * API uni test sketch * Working example of groovy inference with open ai api * Added lines to test * Set default to mpt 2023-06-28 18:28:52 +00:00
			`services:`
			`gpt4all_api:`
			`image: gpt4all_api`
			`container_name: gpt4all_api`
			`restart: always #restart on error (usually code compilation from save during bad state)`
			`ports:`
			`- "4891:4891"`
			`environment:`
			`- APP_ENVIRONMENT=dev`
			`- WEB_CONCURRENCY=2`
			`- LOGLEVEL=debug`
			`- PORT=4891`
			`- model=ggml-mpt-7b-chat.bin`
GPU Inference Server (#1112) * feat: local inference server * fix: source to use bash + vars * chore: isort and black * fix: make file + inference mode * chore: logging * refactor: remove old links * fix: add new env vars * feat: hf inference server * refactor: remove old links * test: batch and single response * chore: black + isort * separate gpu and cpu dockerfiles * moved gpu to separate dockerfile * Fixed test endpoints * Edits to API. server won't start due to failed instantiation error * Method signature * fix: gpu_infer * tests: fix tests --------- Co-authored-by: Andriy Mulyar <andriy.mulyar@gmail.com> 2023-07-21 19:13:29 +00:00			`- inference_mode=cpu`
GPT4All API Scaffolding. Matches OpenAI OpenAPI spec for chats and completions (#839) * GPT4All API Scaffolding. Matches OpenAI OpenAI spec for engines, chats and completions * Edits for docker building * FastAPI app builds and pydantic models are accurate * Added groovy download into dockerfile * improved dockerfile * Chat completions endpoint edits * API uni test sketch * Working example of groovy inference with open ai api * Added lines to test * Set default to mpt 2023-06-28 18:28:52 +00:00			`volumes:`
			`- './gpt4all_api/app:/app'`
			`command: ["/start-reload.sh"]`