Mostofa Patwary

Mostofa Patwary is the Director of Large Foundational Language Model, Applied Deep Learning Research at NVIDIA, where they lead a team focused on pretraining large language models and developing innovative tools for data quality and model accuracy. They previously held roles as a Principal Research Scientist and Senior Engineering Manager at NVIDIA, as well as a Senior Research Scientist at Baidu USA. Mostofa holds a PhD in Computer Science from the University of Bergen and has extensive experience in parallel algorithms, high-performance computing, and deep learning across various prestigious institutions. Their notable publications include work on the Megatron-Turing NLG 530B model and advancements in natural language processing and generative dialog modeling.