Commit Graph

140 Commits

Author SHA1 Message Date
Zach Nussbaum
5de765410a fix: add wd + min lr to config 2023-04-08 20:37:51 +00:00
Zach Nussbaum
c0a9065032 fix: tokenization error 2023-04-08 20:33:51 +00:00
Zach Nussbaum
df29c62521 fix: config + ignore pkl 2023-04-08 20:33:02 +00:00
zanussbaum
fbb788e451 feat: lora gptj 2023-04-07 17:53:07 -04:00
zanussbaum
dcc5204352 fix: batch size 2023-04-07 17:41:45 -04:00
zanussbaum
aa8dd7a636 feat: update for mosaic 2023-04-07 16:54:29 -04:00
Zach Nussbaum
1b5e660476 fix: pyarrow filter 2023-04-07 19:04:19 +00:00
Zach Nussbaum
f974ca651c fix: inference save shards 2023-04-07 16:23:34 +00:00
Zach
573272ad69 fix: drop uneven batch size 2023-04-07 12:09:31 +00:00
Zach
d93f6c2ced fix: concat 2023-04-07 04:33:34 +00:00
Zach
57eb786756 fix: data for inference 2023-04-07 01:45:07 +00:00
Zach
3f9fea47d7 feat: inference for embedding plots 2023-04-07 01:40:39 +00:00
Zach
8f2eb1a583 fix: grad accum loss calc 2023-04-06 12:11:10 +00:00
Zach
e4e88dff33 fix: data processing 2023-04-06 03:03:34 +00:00
Zach
c2fc164779 fix: configs 2023-04-05 20:42:35 +00:00
Zach
838b19bea5 fix: try except push 2023-04-05 20:42:22 +00:00
Zach Nussbaum
885b7f1a3a feat: multinode setup 2023-04-05 02:53:04 +00:00
Zach Nussbaum
98ceac34e1 fix: gptj multinode 2023-04-05 02:52:44 +00:00
Zach Nussbaum
0c8666f635 fix: ignore env 2023-04-05 02:52:21 +00:00
Zach Nussbaum
9567f445b8 fix: only on first process, not once on every node 2023-04-05 02:36:22 +00:00
Zach Nussbaum
ad33b83a48 fix: eval func 2023-04-04 23:25:37 +00:00
Zach
8dd99cc00a fix: prompt len for larger 2023-04-04 22:01:55 +00:00
Zach Nussbaum
63ff39653d feat: gpt-j config 2023-04-04 20:58:08 +00:00
Zach Nussbaum
32357c920f feat: adamw, fix training, log gradients 2023-04-04 20:57:42 +00:00
Zach Nussbaum
c68311810a fix: clean up data, pad at end 2023-04-04 20:53:23 +00:00
Zach Nussbaum
f45eb001a1 fix: clean where prompt is randomly 1 char 2023-04-04 20:47:21 +00:00
Zach Nussbaum
c7f4ca25e4 chore: gitignore ckpts 2023-04-04 20:46:57 +00:00
Andriy Mulyar
9d30a6063e Merge pull request #3 from nomic-ai/train
log wandb multi-epoch
2023-03-29 13:50:26 -04:00
Andriy Mulyar
0eef27364d Qualified number of epochs for LoRa weights 2023-03-29 12:26:47 -04:00
Andriy Mulyar
d61f0077ee Merge pull request #42 from tiendung/main
Change git clone url to https://github.com/nomic-ai/gpt4all.git to avoid `Permission denied`
2023-03-29 10:50:44 -04:00
Andriy Mulyar
4ebac17592 Added Torrent Magnet Link 2023-03-29 10:47:19 -04:00
Andriy Mulyar
f1af332b22 Merge pull request #36 from Hello1024/patch-1
Update README.md to add torrent link to data
2023-03-29 10:38:26 -04:00
Andriy Mulyar
1e0eb13ce7 Merge branch 'main' into patch-1 2023-03-29 10:38:17 -04:00
Andriy Mulyar
8f5067fdca Merge pull request #31 from EliasVincent/chat-windows-binary
Add chat binary for Windows
2023-03-29 10:35:38 -04:00
Andriy Mulyar
299de137a4 Merge branch 'main' into chat-windows-binary 2023-03-29 10:35:31 -04:00
Andriy Mulyar
e642586988 Merge pull request #26 from dsernst/mac-intel-bin
Add binary for Intel Macs
2023-03-29 10:34:52 -04:00
Alex Nguyen
13e61f0123 Change git clone url to https://github.com/nomic-ai/gpt4all.git
git@github.com:nomic-ai/gpt4all.git yield `Permission denied`

```sh
git clone --recurse-submodules git@github.com:nomic-ai/gpt4all.git

Cloning into 'gpt4all'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
```
2023-03-29 20:59:13 +07:00
Andriy Mulyar
9425a0cb46 Changed hosting 2023-03-29 09:55:18 -04:00
Hello1024
3d7cd91042 Update README.md to add torrent link to data 2023-03-29 13:49:32 +01:00
EliasVincent
c8e7b23cfa Add chat binary for Windows 2023-03-29 12:48:13 +02:00
David Ernst
52962f6f1e Add binary for Intel Macs 2023-03-29 03:18:23 -05:00
Zach Nussbaum
668c71dc90 Update data.py 2023-03-28 21:13:05 -07:00
Zach Nussbaum
e220c2f6c8 Update generate_large_2.yaml 2023-03-28 21:03:10 -07:00
Zach Nussbaum
870bd36e27 Update generate_large_3.yaml 2023-03-28 21:02:58 -07:00
Zach Nussbaum
382f35480a Update generate_full.yaml 2023-03-28 21:02:32 -07:00
Zach Nussbaum
6c418d263d Update generate_baseline.yaml 2023-03-28 21:00:57 -07:00
Zach Nussbaum
5a96e50eec Update generate.yaml 2023-03-28 21:00:36 -07:00
Zach Nussbaum
ab4b13b2fa Update finetune_lora.yaml 2023-03-28 20:58:33 -07:00
Zach Nussbaum
b0fd406ebb Update finetune.yaml 2023-03-28 20:58:03 -07:00
Andriy Mulyar
b5c5857241 Merge pull request #16 from mazzzystar/main
To make the model bin file more clear in README.
2023-03-28 22:43:03 -04:00