PhD Course: Large language models: from BERT to GPT with reinforcement learning

The lectures of the PhD course "Large language models: from BERT to GPT with reinforcement learning" by Prof. Andrea Tagarelli will be held at the DIMES Seminar Room (Cube 42C, 5th Floor) on March 20th and 21st, 09:00 - 13:00.

Abstract of the course (8h):

Research in artificial intelligence for natural language processing (NLP) has a long timeline, spanning over 70 years and characterized by several milestones, peaking with the recent pre-trained deep contextualized languagemodels based on Transformers. Such models have gained tremendous success, bringing significant performance boosts in a wide range of tasks and benchmarks in NLP. Also, the principles underlying Transformers and their deep-learning architectural traits are used in the current state-of-the-art of Computer Vision and Speech Processing. 
In this context, the proposed course aims to provide an analysis of the key concepts and neural architectures that characterize the Transformer-based languagemodels, with particular emphasis on the BERTmodels and on the generative Transformers, namely the GPT and InstructGPT family, which include the more recent and revolutionary ChatGPT. The course also offers a glimpse into future perspectives and new opportunities of artificial intelligence for supporting various application fields, possibly of high societal impact, such as law and healthcare.