9 Commits

Author	SHA1	Message	Date
AT	bbe195ee02	Backend prompt dedup (#822) ... * Deduplicated prompt() function code	2023-06-04 08:59:24 -04:00
Adam Treat	cec8831e12	Fix mac build again.	2023-06-02 10:51:09 -04:00
Adam Treat	70e3b7e907	Try and fix build on mac.	2023-06-02 10:47:12 -04:00
AT	48275d0dcc	Dlopen backend 5 (#779) ... Major change to the backend that allows for pluggable versions of llama.cpp/ggml. This was squashed merged from dlopen_backend_5 where the history is preserved.	2023-05-31 17:04:01 -04:00
Adam Treat	7f9f91ad94	Revert "New tokenizer implementation for MPT and GPT-J" ... This reverts commit bbcee1ced5.	2023-05-30 12:59:00 -04:00
Aaron Miller	bbcee1ced5	New tokenizer implementation for MPT and GPT-J ... Improves output quality by making these tokenizers more closely match the behavior of the huggingface `tokenizers` based BPE tokenizers these models were trained with. Featuring: * Fixed unicode handling (via ICU) * Fixed BPE token merge handling * Complete added vocabulary handling	2023-05-30 12:05:57 -04:00
Adam Treat	474c5387f9	Get the backend as well as the client building/working with msvc.	2023-05-25 15:22:45 -04:00
kuvaus	3cb6dd7a66	gpt4all-backend: Add llmodel create and destroy functions (#554) ... * Add llmodel create and destroy functions * Fix capitalization * Fix capitalization * Fix capitalization * Update CMakeLists.txt --------- Co-authored-by: kuvaus <kuvaus@users.noreply.github.com>	2023-05-16 11:36:46 -04:00
Adam Treat	d918b02c29	Move the llmodel C API to new top-level directory and version it.	2023-05-10 11:46:40 -04:00