I want airdialogue prepro to support an alternate inference input mode for dialogue data.

I want airdialogue prepro to support an alternate inference input mode for dialogue data.: a task in MiMo-V2.6-RL-harbor-code: MiMo-V2.6-RL Code (Harbor) (Harbor dataset). It should keep the existing --keep incorrect behavior: by default skip records whose JSON has correct sample: false , and…

The task

It should keep the existing `--keep_incorrect` behavior: by default skip records whose JSON has `correct_sample: false`, and include them when `--keep_incorrect true` is supplied. It should remove non-ASCII characters before JSON parsing and skip very short padding lines in both alternate input files. For every…

Part of FineEnvs/MiMo-V2.6-RL-harbor-code.