gpt4all/gpt4all-api/gpt4all_api
Zach Nussbaum 8aba2c9009
GPU Inference Server (#1112)
* feat: local inference server

* fix: source to use bash + vars

* chore: isort and black

* fix: make file + inference mode

* chore: logging

* refactor: remove old links

* fix: add new env vars

* feat: hf inference server

* refactor: remove old links

* test: batch and single response

* chore: black + isort

* separate gpu and cpu dockerfiles

* moved gpu to separate dockerfile

* Fixed test endpoints

* Edits to API. server won't start due to failed instantiation error

* Method signature

* fix: gpu_infer

* tests: fix tests

---------

Co-authored-by: Andriy Mulyar <andriy.mulyar@gmail.com>
2023-07-21 15:13:29 -04:00
..
app GPU Inference Server (#1112) 2023-07-21 15:13:29 -04:00
Dockerfile.buildkit GPT4All API Scaffolding. Matches OpenAI OpenAPI spec for chats and completions (#839) 2023-06-28 14:28:52 -04:00
README.md GPT4All API Scaffolding. Matches OpenAI OpenAPI spec for chats and completions (#839) 2023-06-28 14:28:52 -04:00
requirements.txt GPU Inference Server (#1112) 2023-07-21 15:13:29 -04:00

FastAPI app for serving GPT4All models