13 Commits

Author	SHA1	Message	Date
Zach Nussbaum	6e84421f3f	fix: num training steps for lr decay	2023-04-10 02:15:31 +00:00
Zach Nussbaum	6de58dd1fc	fix: saving name	2023-04-08 20:56:13 +00:00
Zach Nussbaum	c1ca081cda	fix: eos/pad token + wd	2023-04-08 20:38:10 +00:00
Zach	8f2eb1a583	fix: grad accum loss calc	2023-04-06 12:11:10 +00:00
Zach	838b19bea5	fix: try except push	2023-04-05 20:42:22 +00:00
Zach Nussbaum	9567f445b8	fix: only on first process, not once on every node	2023-04-05 02:36:22 +00:00
Zach Nussbaum	ad33b83a48	fix: eval func	2023-04-04 23:25:37 +00:00
Zach Nussbaum	32357c920f	feat: adamw, fix training, log gradients	2023-04-04 20:57:42 +00:00
Zach Nussbaum	812b807003	fix: log for multiple epochs	2023-03-28 18:47:58 +00:00
Zach Nussbaum	b24ca03dc4	fix: append first_epoch	2023-03-27 17:30:24 +00:00
Zach Nussbaum	24765a1965	fix: add epoch train	2023-03-27 16:32:35 +00:00
Zach Nussbaum	2daecd6066	feat: update embeddings	2023-03-26 17:45:21 +00:00
Zach Nussbaum	723a50bdf1	feat: train and clean data	2023-03-25 16:17:48 +00:00