When I call dask.array.from array() on a large numpy.memmap , Dask seems to touch the whole mapped file…
When I call dask.array.from array() on a large numpy.memmap , Dask seems to touch the whole mapped file…: a task in MiMo-V2.6-RL-harbor-code: MiMo-V2.6-RL Code (Harbor) (Harbor dataset). Expected outcomes: - dask.base.tokenize(...) handles numpy.memmap inputs without materializing or hashing the…
The task
Expected outcomes: - `dask.base.tokenize(...)` handles `numpy.memmap` inputs without materializing or hashing the full mapped array contents. - `dask.array.from_array(...)` can build a graph from a large `numpy.memmap` without paging the whole backing file into memory during graph construction. - Tokens for…
Part of FineEnvs/MiMo-V2.6-RL-harbor-code.