From 8ac7c1a9fe623af38413519c06f05f46fb3358dd Mon Sep 17 00:00:00 2001 From: Ikko Eltociear Ashimine Date: Fri, 31 Mar 2023 00:53:53 +0900 Subject: [PATCH] Fix typo in TRAINING_LOG.md Conditonal -> Conditional --- TRAINING_LOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TRAINING_LOG.md b/TRAINING_LOG.md index 31b9bb21..50469645 100644 --- a/TRAINING_LOG.md +++ b/TRAINING_LOG.md @@ -160,7 +160,7 @@ We realized that we had two bugs however: - We accidentally duplicated data and effectively trained for 2 epochs instead of 1 - We added an eos token to every sequence, even those that we truncated (e.g. long code that exceeds the 1024). -## Conditonal EOS and 1 Epoch +## Conditional EOS and 1 Epoch Using the same parameters, we then trained a model using a "conditional" eos token where we only add an `eos` when the inputs are less than the maximum sequence length for one epoch.