Exploring Transformer Architectures and Model Pruning in Generative Modeling

This chapter examines the practical deployment of transformer architectures of varying sizes, emphasizing the impacts of model pruning on performance. It compares decoder-only transformers with architectures like BERT and pixel CNN to understand their effectiveness in generative modeling.