Repair effective-batch normalization in a frozen Transformers label-smoothing path and produce a…
Repair effective-batch normalization in a frozen Transformers label-smoothing path and produce a…: a task in MiMo-V2.6-RL-harbor-terminal: MiMo-V2.6-RL Terminal (Harbor) (Harbor dataset). A deterministic language-model training reproduction in /app/vendor/transformers/tools/effective batch case.py…
The task
A deterministic language-model training reproduction in `/app/vendor/transformers/tools/effective_batch_case.py` exposes a regression in the existing Trainer label-smoothing path. With gradient accumulation, microbatches can contain different numbers of non-ignored target tokens. Repair the existing framework…