Foundations of Artificial Intelligence Seminar Series

📝Abstract General intelligences, both human and artificial (e.g. LLMs), offer remarkable flexibility in handling diverse tasks. However, directly leveraging these general intelligences at scale is prohibitively expensive. This raises the key question of how we can efficiently train lightweight, specialized models for specific applications by learning from and distilling the knowledge of black-box general intelligences. In this talk, I will discuss the label-efficient learning paradigms that have been developed over the past two decades, covering techniques in active learning, semi-supervised learning, and transfer learning. I will highlight scenarios and approaches that have proven empirically effective for label-efficient learning, including fine-tuning large pretrained models, uncertainty sampling and handling class imbalance. I will conclude by discussing the challenges and growing importance of label-efficient learning in an open-world scenario. This talk will provide an overview of the key ideas, results, and open problems in learning efficiently from black-box general intelligences.