In the ninth ALIGN webinar, we invited Dr. Albert Webson, a senior research scientist at Google DeepMind. During his PhD at Brown University, he co-first-authored several early instruction-tuned models, such as T0, FLAN-T5 and BLOOM. At DeepMind, he is one of the main technical contributors to Gemini pretraining and reinforcement learning.
He gave an overview of the main components of contemporary LLM pretraining and post-training (SFT and RLHF), including their connections to the history of alignment, followed by questions and discussion with participants. The talk was in English with slides in Japanese, and questions could also be asked in Japanese. The talk is not available on our YouTube channel.