Papers Read on AI cover image

DoRA: Weight-Decomposed Low-Rank Adaptation

Papers Read on AI

00:00

SVD Decomposition and Low-Rank Adaptation in Federated Learning

Exploring the use of SVD decomposition for singular value pruning, low-rank adaptation in federated learning, orthogonal factorization, weight tying, stable diffusion framework, and learnable scaling vectors for random matrices. The chapter also delves into weight decomposition analysis, comparing Laura to Fort for expressiveness and reveals differences in learning patterns through restructuring weight matrices.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app