My research lies in the broad area of artificial intelligence and machine learning. I am particularly interested in generative models of high dimensional data, and in designing principled methods for improved robustness and generalization in machine learning. More recently, I have been working on large language models, studying attention biases in transformers, efficiency aspects in LLMs through distillation and activation sparsity, and alternative architectures for sequence modeling and their scaling properties.