Repair hierarchical ML evaluator aggregation and produce an offline integration artifact.
Repair hierarchical ML evaluator aggregation and produce an offline integration artifact.: a task in MiMo-V2.6-RL-harbor-terminal: MiMo-V2.6-RL Terminal (Harbor) (Harbor dataset). The evaluator in /app/vendor/lm-evaluation-harness has a regression in hierarchical metric aggregation. Diagnose the…
The task
The evaluator in `/app/vendor/lm-evaluation-harness` has a regression in hierarchical metric aggregation. Diagnose the interaction between `weight_by_size`, point estimates, and reported standard errors, and repair the existing production modules in `lm_eval/api/metrics.py` and `lm_eval/api/group.py`.