Fix a regression in FullyShardedDataParallelPlugin.set auto wrap policy : transformer-based auto-wrapping…
Fix a regression in FullyShardedDataParallelPlugin.set auto wrap policy : transformer-based auto-wrapping…: a task in HF ML Tasksmith (Harbor dataset). Example that must work: a model with no split modules = "MyTransformerBlock" passed to set auto wrap policy with transformer cls names to…
The task
**Example that must work:** a model with `_no_split_modules = ["MyTransformerBlock"]` passed to `set_auto_wrap_policy` with `transformer_cls_names_to_wrap=None` should resolve `"MyTransformerBlock"` as the class to wrap and produce a `functools.partial` as `auto_wrap_policy`. It must **not** attempt to look up…
Part of FineEnvs/HF_ML_Tasksmith.