LoRA Wrapper for Existing Models

[Glar35]

Does a LoRA (Low-Rank Adaptation) wrapper for existing models already exist? If not, is it technically possible to construct one?

[laggui]

There is no official implementation at this time 🙂 LoRA is a simple re-parametrization of the linear layer weights, so this should be possible to implement.

We'd like to have LoRA with burn for users to leverage easily, so it's probably gonna appear at some point!