161 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Zach Nussbaum	855bc17b1d	fix: gpt-j data link	2023-04-13 17:41:07 +00:00
Andriy Mulyar	5467ec78ea	Update README.md	2023-04-13 12:57:12 -04:00
Andriy Mulyar	053f2e4f24	Update README.md	2023-04-13 12:56:08 -04:00
Zach Nussbaum	d7395ee37a	Merge: main into gptj	2023-04-13 15:16:31 +00:00
Zach Nussbaum	b1e361882d	fix: multi-turn data breaks	2023-04-12 03:51:29 +00:00
Zach Nussbaum	60155de2a6	chore: peft	2023-04-12 03:50:54 +00:00
Zach Nussbaum	3c11e68ec8	feat: commits for eval + generation	2023-04-11 19:14:29 +00:00
Zach Nussbaum	70d75951d1	chore: remove not needed	2023-04-11 12:39:07 +00:00
Zach Nussbaum	84de913baf	chore: submodule ff	2023-04-10 02:16:05 +00:00
Zach Nussbaum	f2cb38e65a	Merge branch 'gptj' of github.com:nomic-ai/gpt4all into gptj	2023-04-10 02:15:47 +00:00
Zach Nussbaum	6e84421f3f	fix: num training steps for lr decay	2023-04-10 02:15:31 +00:00
Zach	8733576664	feat: evals on new gptj models	2023-04-10 02:14:20 +00:00
Zach	5caed073ee	fix: topic model for embeddings	2023-04-09 15:12:49 +00:00
Zach Nussbaum	c11ea7dc31	fix: bs try one more time?	2023-04-08 21:47:07 +00:00
Zach Nussbaum	2827693a57	fix: smaller bs for 40gb	2023-04-08 21:36:20 +00:00
Zach Nussbaum	f5ca8bedcf	fix: config for lora gptj	2023-04-08 21:17:12 +00:00
Zach Nussbaum	6de58dd1fc	fix: saving name	2023-04-08 20:56:13 +00:00
Zach Nussbaum	305fe3d444	Merge remote-tracking branch 'origin/mosaic' into gptj	2023-04-08 20:47:01 +00:00
Zach Nussbaum	c1ca081cda	fix: eos/pad token + wd	2023-04-08 20:38:10 +00:00
Zach Nussbaum	5de765410a	fix: add wd + min lr to config	2023-04-08 20:37:51 +00:00
Zach Nussbaum	c0a9065032	fix: tokenization error	2023-04-08 20:33:51 +00:00
Zach Nussbaum	df29c62521	fix: config + ignore pkl	2023-04-08 20:33:02 +00:00
Zach Nussbaum	f479a37818	feat: build map script	2023-04-08 19:30:53 +00:00
Zach	2034f0c479	fix: embeddings instead of logits!!!	2023-04-08 17:05:40 +00:00
zanussbaum	fbb788e451	feat: lora gptj	2023-04-07 17:53:07 -04:00
zanussbaum	dcc5204352	fix: batch size	2023-04-07 17:41:45 -04:00
zanussbaum	aa8dd7a636	feat: update for mosaic	2023-04-07 16:54:29 -04:00
Zach Nussbaum	1b5e660476	fix: pyarrow filter	2023-04-07 19:04:19 +00:00
Zach Nussbaum	f974ca651c	fix: inference save shards	2023-04-07 16:23:34 +00:00
Zach	573272ad69	fix: drop uneven batch size	2023-04-07 12:09:31 +00:00
Zach	d93f6c2ced	fix: concat	2023-04-07 04:33:34 +00:00
Zach	57eb786756	fix: data for inference	2023-04-07 01:45:07 +00:00
Zach	3f9fea47d7	feat: inference for embedding plots	2023-04-07 01:40:39 +00:00
Zach	8f2eb1a583	fix: grad accum loss calc	2023-04-06 12:11:10 +00:00
Zach	e4e88dff33	fix: data processing	2023-04-06 03:03:34 +00:00
Zach	c2fc164779	fix: configs	2023-04-05 20:42:35 +00:00
Zach	838b19bea5	fix: try except push	2023-04-05 20:42:22 +00:00
Zach Nussbaum	885b7f1a3a	feat: multinode setup	2023-04-05 02:53:04 +00:00
Zach Nussbaum	98ceac34e1	fix: gptj multinode	2023-04-05 02:52:44 +00:00
Zach Nussbaum	0c8666f635	fix: ignore env	2023-04-05 02:52:21 +00:00
Zach Nussbaum	9567f445b8	fix: only on first process, not once on every node	2023-04-05 02:36:22 +00:00
Zach Nussbaum	ad33b83a48	fix: eval func	2023-04-04 23:25:37 +00:00
Zach	8dd99cc00a	fix: prompt len for larger	2023-04-04 22:01:55 +00:00
Zach Nussbaum	63ff39653d	feat: gpt-j config	2023-04-04 20:58:08 +00:00
Zach Nussbaum	32357c920f	feat: adamw, fix training, log gradients	2023-04-04 20:57:42 +00:00
Zach Nussbaum	c68311810a	fix: clean up data, pad at end	2023-04-04 20:53:23 +00:00
Zach Nussbaum	f45eb001a1	fix: clean where prompt is randomly 1 char	2023-04-04 20:47:21 +00:00
Zach Nussbaum	c7f4ca25e4	chore: gitignore ckpts	2023-04-04 20:46:57 +00:00
Andriy Mulyar	9d30a6063e	Merge pull request #3 from nomic-ai/train ... log wandb multi-epoch	2023-03-29 13:50:26 -04:00
Andriy Mulyar	0eef27364d	Qualified number of epochs for LoRa weights	2023-03-29 12:26:47 -04:00