Add Generalized Low Rank Adaptation (GLoRA) to PEFT for ordinary torch.nn.Linear layers. Expose GloraConfig…
Add Generalized Low Rank Adaptation (GLoRA) to PEFT for ordinary torch.nn.Linear layers. Expose GloraConfig…: a task in HF ML Tasksmith (Harbor dataset). For a frozen base weight W and bias b, the adapter's effective weight is W + W A + B (elementwise multiplication). If the original layer has a…
The task
For a frozen base weight W and bias b, the adapter's effective weight is W + W*A + B (elementwise multiplication). If the original layer has a bias, its effective bias is b + b*D + E + W@C. Preserve ordinary linear input/output shape semantics. Bias-free layers receive the A/B weight correction only; do not invent a…
Part of FineEnvs/HF_ML_Tasksmith.