Fix three bugs in OnlineDPOTrainer. generate vllm server so that it produces the same block-layout batch…
Fix three bugs in OnlineDPOTrainer. generate vllm server so that it produces the same block-layout batch…: a task in HF ML Tasksmith (Harbor dataset). Background. Online DPO generates exactly 2 completions per prompt to form preference pairs. Everything downstream ( rewards.split(batch size) )…
The task
**Background.** Online DPO generates exactly 2 completions per prompt to form preference pairs. Everything downstream (`rewards.split(batch_size)`) expects a **block layout**: rows 0…N-1 are "completion 0" for each prompt, rows N…2N-1 are "completion 1". Online DPO does **not** use a repeat sampler; prompts in the…
Part of FineEnvs/HF_ML_Tasksmith.