gpt4all

Commit Graph

Author	SHA1	Message	Date
Jared Van Bortel	be66ec8ab5	chat: faster KV shift, continue generating, fix stop sequences (#2781 ) * Don't stop generating at end of context * Use llama_kv_cache ops to shift context * Fix and improve reverse prompt detection * Replace prompt recalc callback with a flag to disallow context shift	1 month ago
Jared Van Bortel	51bd01ae05	backend: fix extra spaces in tokenization and a CUDA crash (#2778 ) Also potentially improves accuracy of BOS insertion, token cache, and logit indexing. Signed-off-by: Jared Van Bortel <jared@nomic.ai>	2 months ago
Jared Van Bortel	01870b4a46	chat: fix blank device in UI and improve Mixpanel reporting (#2409 ) Also remove LLModel::hasGPUDevice. Signed-off-by: Jared Van Bortel <jared@nomic.ai>	3 months ago
Jared Van Bortel	636307160e	backend: fix #includes with include-what-you-use (#2371 ) Also fix a PARENT_SCOPE warning when building the backend. Signed-off-by: Jared Van Bortel <jared@nomic.ai>	4 months ago
Jared Van Bortel	d2a99d9bc6	support the llama.cpp CUDA backend (#2310 ) * rebase onto llama.cpp commit ggerganov/llama.cpp@d46dbc76f * support for CUDA backend (enabled by default) * partial support for Occam's Vulkan backend (disabled by default) * partial support for HIP/ROCm backend (disabled by default) * sync llama.cpp.cmake with upstream llama.cpp CMakeLists.txt * changes to GPT4All backend, bindings, and chat UI to handle choice of llama.cpp backend (Kompute or CUDA) * ship CUDA runtime with installed version * make device selection in the UI on macOS actually do something * model whitelist: remove dbrx, mamba, persimmon, plamo; add internlm and starcoder2 Signed-off-by: Jared Van Bortel <jared@nomic.ai>	4 months ago
Jared Van Bortel	ba53ab5da0	python: do not print GPU name with verbose=False, expose this info via properties (#2222 ) * llamamodel: only print device used in verbose mode Signed-off-by: Jared Van Bortel <jared@nomic.ai> * python: expose backend and device via GPT4All properties Signed-off-by: Jared Van Bortel <jared@nomic.ai> * backend: const correctness fixes Signed-off-by: Jared Van Bortel <jared@nomic.ai> * python: bump version Signed-off-by: Jared Van Bortel <jared@nomic.ai> * python: typing fixups Signed-off-by: Jared Van Bortel <jared@nomic.ai> * python: fix segfault with closed GPT4All Signed-off-by: Jared Van Bortel <jared@nomic.ai> --------- Signed-off-by: Jared Van Bortel <jared@nomic.ai>	5 months ago
Jared Van Bortel	46818e466e	python: embedding cancel callback for nomic client dynamic mode (#2214 ) Signed-off-by: Jared Van Bortel <jared@nomic.ai>	5 months ago
Jared Van Bortel	0455b80b7f	Embed4All: optionally count tokens, misc fixes (#2145 ) Key changes: * python: optionally return token count in Embed4All.embed * python and docs: models2.json -> models3.json * Embed4All: require explicit prefix for unknown models * llamamodel: fix shouldAddBOS for Bert and Nomic Bert Signed-off-by: Jared Van Bortel <jared@nomic.ai>	6 months ago
Jared Van Bortel	406e88b59a	implement local Nomic Embed via llama.cpp (#2086 ) Signed-off-by: Jared Van Bortel <jared@nomic.ai>	6 months ago
Jared Van Bortel	4fc4d94be4	fix chat-style prompt templates (#1970 ) Also use a new version of Mistral OpenOrca. Signed-off-by: Jared Van Bortel <jared@nomic.ai>	7 months ago
Jared Van Bortel	061d1969f8	expose n_gpu_layers parameter of llama.cpp (#1890 ) Also dynamically limit the GPU layers and context length fields to the maximum supported by the model. Signed-off-by: Jared Van Bortel <jared@nomic.ai>	8 months ago
Jared Van Bortel	38c61493d2	backend: update to latest commit of llama.cpp Vulkan PR Signed-off-by: Jared Van Bortel <jared@nomic.ai>	8 months ago
Jared Van Bortel	d1c56b8b28	Implement configurable context length (#1749 )	9 months ago
Cebtenzzre	672cb850f9	differentiate between init failure and unsupported models	12 months ago
Adam Treat	3076e0bf26	Only show GPU when we're actually using it.	1 year ago
Adam Treat	987546c63b	Nomic vulkan backend licensed under the Software for Open Models License (SOM), version 1.0.	1 year ago
Adam Treat	0efdbfcffe	Bert	1 year ago
Aaron Miller	b19a3e5b2c	add requiredMem method to llmodel impls most of these can just shortcut out of the model loading logic llama is a bit worse to deal with because we submodule it so I have to at least parse the hparams, and then I just use the size on disk as an estimate for the mem size (which seems reasonable since we mmap() the llama files anyway)	1 year ago
Aaron Miller	88616fde7f	llmodel: change tokenToString to not use string_view (#968 ) fixes a definite use-after-free and likely avoids some other potential ones - std::string will convert to a std::string_view automatically but as soon as the std::string in question goes out of scope it is already freed and the string_view is pointing at freed memory - this is mostly fine if its returning a reference to the tokenizer's internal vocab table but it's, imo, too easy to return a reference to a dynamically constructed string with this as replit is doing (and unfortunately needs to do to convert the internal whitespace replacement symbol back to a space)	1 year ago
Adam Treat	301d2fdbea	Fix up for newer models on reset context. This fixes the model from totally failing after a reset context.	1 year ago
AT	bbe195ee02	Backend prompt dedup (#822 ) * Deduplicated prompt() function code	1 year ago
Adam Treat	a41bd6ac0a	Trying to shrink the copy+paste code and do more code sharing between backend model impl.	1 year ago
AT	48275d0dcc	Dlopen backend 5 (#779 ) Major change to the backend that allows for pluggable versions of llama.cpp/ggml. This was squashed merged from dlopen_backend_5 where the history is preserved.	1 year ago

23 Commits (b70d68977db50c1266c80129dcc1cbb0080fd4d6)