Mathematical Theory for Emergent Intelligence

Workshop Website

http://ias.ust.hk/events/202307matei/
 

Overview

By any measure the success of contemporary approaches to artificial intelligence has been stunning. While the rise of neural networks, deep learning, and large language models has led to applications unthinkable even a few short years ago, many puzzles remain. Most perplexing perhaps is the deep question of *why* these models have proven so successful. This workshop intends to bring together mathematicians, computer scientists, and other researchers to explore this and related questions from a mathematical perspective. For instance,

• Why did intelligence seem to appear when LLMs are big enough?

• How big is big enough? 175B parameters (OpenAI GPT3)?

• The intelligence “emerged” from GPT2 to GPT3 when model scaled 2 orders of magnitude. Will the same happened when we scale another 2 orders in model parameters from GPT3?

• What kinds of training data will help intelligence “emerge”? Coding data?

• Why are we still looking for “theory of anything” for deep learning especially with LLMs?

• What might be the mathematical principles behind the emergent intelligence?

• What are the right mathematical tools to study the emergent intelligence?

We hope to study these and many relevant and exciting problems at the workshop. Send us your questions and join us at HKUST for the workshop!

Organizers

• Harry SHUM, HKUST (IAS Professor-at-Large Emeritus)

• Yi MA, The University of Hong Kong and University of California, Berkeley

• Andrew COHEN, HKUST (IAS Director, IAS Professor, and Lam Woo Foundation Professor)

• Qifeng CHEN, HKUST

Sponsor