Archives/gpt4all
2
0
Fork 0
mirror of https://github.com/nomic-ai/gpt4all synced 2024-11-06 09:20:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix typo in TRAINING_LOG.md

Browse Source 
Conditonal -> Conditional
This commit is contained in:
Ikko Eltociear Ashimine 2023-03-31 00:53:53 +09:00 committed by GitHub
parent f7d4b4bde7
commit 8ac7c1a9fe
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
TRAINING_LOG.md
View File

@ -160,7 +160,7 @@ We realized that we had two bugs however:
- We accidentally duplicated data and effectively trained for 2 epochs instead of 1 - We accidentally duplicated data and effectively trained for 2 epochs instead of 1
- We added an eos token to every sequence, even those that we truncated (e.g. long code that exceeds the 1024). - We added an eos token to every sequence, even those that we truncated (e.g. long code that exceeds the 1024).
## Conditonal EOS and 1 Epoch ## Conditional EOS and 1 Epoch
Using the same parameters, we then trained a model using a "conditional" eos token where we only add an `eos` when the inputs are less than the maximum sequence length for one epoch. Using the same parameters, we then trained a model using a "conditional" eos token where we only add an `eos` when the inputs are less than the maximum sequence length for one epoch.