Archives/gpt4all
2
0
Fork 0
mirror of https://github.com/nomic-ai/gpt4all synced 2024-11-06 09:20:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
891 Commits 540 Branches 59 Tags 51 MiB
15c6bf09e9
Commit Graph

891 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
aaron miller
9c15d1f83e add tokenizer readme w/ instructions for convert script 2023-05-30 12:05:57 -04:00
Aaron Miller
840e011b75 buf_ref.into() can be const now 2023-05-30 12:05:57 -04:00
Aaron Miller
ee3469ba6c New tokenizer implementation for MPT and GPT-J 
Improves output quality by making these tokenizers more closely
match the behavior of the huggingface `tokenizers` based BPE
tokenizers these models were trained with.

Featuring:
 * Fixed unicode handling (via ICU)
 * Fixed BPE token merge handling
 * Complete added vocabulary handling
 2023-05-30 12:05:57 -04:00
Andriy Mulyar
7e18f179e9 Improved localdocs documentation (#762) 
* Improved localdocs documentation

* Improved localdocs documentation

* Improved localdocs documentation

* Improved localdocs documentation
 2023-05-30 11:26:34 -04:00
Andriy Mulyar
3bb19b5e91 LocalDocs documentation initial (#761) 
* LocalDocs documentation initial
 2023-05-30 08:35:26 -04:00
mvenditto
522dfdbfe1 C# Bindings - Prompt formatting (#712) 
* Added support for custom prompt formatting

* more docs added

* bump version
 2023-05-28 19:57:00 -04:00
Chase McDougall
8189aece48 fix(training instructions): model repo name (#728) 
Signed-off-by: Chase McDougall <chasemcdougall@hotmail.com>
 2023-05-28 19:56:24 -04:00
Nandakumar
4161bfba33 Update README.md (#738) 
* Update README.md

fix golang gpt4all import path

Signed-off-by: Nandakumar <nandagunasekaran@gmail.com>

* Update README.md

Signed-off-by: Nandakumar <nandagunasekaran@gmail.com>

---------

Signed-off-by: Nandakumar <nandagunasekaran@gmail.com>
 2023-05-28 19:51:11 -04:00
Joseph Mearman
6ec2e26cd1 tiny typo (#739) 2023-05-28 19:50:45 -04:00
Richard Guo
9f62d4d05f hotfix default verbose optioin 2023-05-26 12:49:32 -04:00
Konstantin Gukov
6ec3efa355 one funcion to append .bin suffix 2023-05-26 09:24:03 -04:00
Konstantin Gukov
52efd17165 Correct indentation of the multiline error message 2023-05-26 09:24:03 -04:00
Konstantin Gukov
39bdd74ddd Add optional verbosity 2023-05-26 09:24:03 -04:00
Konstantin Gukov
136a5ac30d Correct return type 2023-05-26 09:24:03 -04:00
Konstantin Gukov
e362e18431 Do not ignore explicitly passed 4 threads 2023-05-26 09:24:03 -04:00
Konstantin Gukov
a2e2a64899 Redundant else 2023-05-26 09:24:03 -04:00
Konstantin Gukov
8229bb5477 1. Cleanup the interrupted download 
2. with-syntax
 2023-05-26 09:24:03 -04:00
Konstantin Gukov
b12e0e98c7 less magic number 2023-05-26 09:24:03 -04:00
Konstantin Gukov
4dff67cee5 convert to f-strings 2023-05-26 09:24:03 -04:00
Konstantin Gukov
6e3c59d8b2 reduce nesting, better error reporting 2023-05-26 09:24:03 -04:00
Konstantin Gukov
591a047204 Concise model matching 2023-05-26 09:24:03 -04:00
Konstantin Gukov
4de8f991f9 Log where the model was found 2023-05-26 09:24:03 -04:00
Konstantin Gukov
b6b21441c7 Nicer handling of missing model directory. 
Correct exception message.
 2023-05-26 09:24:03 -04:00
Konstantin Gukov
84201784b5 More precise condition 2023-05-26 09:24:03 -04:00
Konstantin Gukov
e3bdc8fc87 rm redundant json 2023-05-26 09:24:03 -04:00
Adam Treat
55b4f35510 This time remember to bump the version right after a release. 2023-05-25 18:26:33 -04:00
Adam Treat
c5c8ab9138 Bump the version number. 2023-05-25 17:08:50 -04:00
Adam Treat
4e8dba107f Libraries named differently on msvc. 2023-05-25 16:27:09 -04:00
Adam Treat
d40735a2d2 Get the backend as well as the client building/working with msvc. 2023-05-25 15:22:45 -04:00
redthing1
0e11584783 make sample print usage and cleaner 2023-05-25 11:34:21 -04:00
redthing1
448acb337d create test project and basic model loading tests 2023-05-25 11:34:07 -04:00
redthing1
9248bd0c1e ignore rider and vscode dirs 2023-05-25 11:34:07 -04:00
Adam Treat
ff51cb872f Add a newline 2023-05-25 11:28:06 -04:00
Adam Treat
580aceb2b3 Various fixes to remove unnecessary warnings. 2023-05-25 11:28:06 -04:00
Adam Treat
e6f5d88f17 Don't use the full path in reference text. 2023-05-25 11:28:06 -04:00
Adam Treat
0a4c3a14ca Add context link to references. 2023-05-25 11:28:06 -04:00
Adam Treat
f3dc031a9d Store the references separately so they are not sent to datalake. 2023-05-25 11:28:06 -04:00
Adam Treat
05c3edb387 Adds the collections to serialize and implement references for localdocs. 2023-05-25 11:28:06 -04:00
Adam Treat
90b2cf747f Complete the settings for localdocs. 2023-05-25 11:28:06 -04:00
Adam Treat
53cf2da5f7 Add more of the UI for selecting collections for chats. 2023-05-25 11:28:06 -04:00
Adam Treat
63c5911604 Clean up the settings dialog for localdocs a bit. 2023-05-25 11:28:06 -04:00
Adam Treat
1930070fdb Begin implementing the localdocs ui in earnest. 2023-05-25 11:28:06 -04:00
Adam Treat
695d1d048b Start fleshing out the localdocs ui. 2023-05-25 11:28:06 -04:00
Adam Treat
38c403dab8 Add a localdocs tab. 2023-05-25 11:28:06 -04:00
Adam Treat
32be0a92d7 Add a collection list to support a UI. 2023-05-25 11:28:06 -04:00
Adam Treat
e47f4ddfb6 Specify a large number of suffixes we will search for now. 2023-05-25 11:28:06 -04:00
Adam Treat
c33bf0e895 Add prompt processing and localdocs to the busy indicator in UI. 2023-05-25 11:28:06 -04:00
Adam Treat
837ece220f Turn off the debugging messages by default. 2023-05-25 11:28:06 -04:00
Adam Treat
90ab9183ae Add a new muted text color. 2023-05-25 11:28:06 -04:00
Adam Treat
80024a029c Add new reverse prompt for new localdocs context feature. 2023-05-25 11:28:06 -04:00