Warmup
Also written: warmup_steps
Starting the learning rate near 0 and raising it over the first steps, so the first noisy updates are small. A value below 1 is a fraction of all steps (0.1 means the first 10%).
Also written: warmup_steps
Starting the learning rate near 0 and raising it over the first steps, so the first noisy updates are small. A value below 1 is a fraction of all steps (0.1 means the first 10%).