undefined

Dan Fu

AI researcher at Together AI, joining UCSD as faculty soon. Contributed to research on post-transformer architectures and efficient kernels.

Best podcasts with Dan Fu

Ranked by the Snipd community
undefined
22 snips
Dec 24, 2024 • 43min

2024 in Post-Transformers Architectures (State Space Models, RWKV) [LS Live @ NeurIPS]

Dan Fu, an AI researcher soon to join UCSD, and Eugene Cheah, CEO of Featherless AI, delve into the future of post-transformer architectures. They discuss innovations like RWKV and state-space models, highlighting their collaborative and open-source nature. The duo examines the challenges of multilingual training and computational efficiency, while also exploring advancements in non-transformer models like Mamba and Jamba. Tune in for insights on scaling models, 'infinite context,' and how new architectures are reshaping the AI landscape!