Towards Provably Efficient Quantum Algorithms for Large-scale Machine-learning Models

Abstract

Large machine learning models are revolutionary technologies of artificial intelligence whose bottlenecks include huge computational expenses, power, and time used both in the pre-training and fine-tuning process. In this work, the speaker’s team show that fault-tolerant quantum computing could possibly provide provably efficient resolutions for generic (stochastic) gradient descent algorithms, scaling as O(T2×polylog(n)), where n is the size of the models and T is the number of iterations in the training, as long as the models are both sufficiently dissipative and sparse. Based on earlier efficient quantum algorithms for dissipative differential equations, the team finds and proves that similar algorithms work for (stochastic) gradient descent, the primary algorithm for machine learning. In practice, the team benchmarks instances of large machine learning models from 7 million to 103 million parameters. The team finds that, in the context of sparse training, a quantum enhancement is possible at the early stage of learning after model pruning, motivating a sparse parameter download and re-upload scheme. The team’s work shows solidly that fault-tolerant quantum algorithms could potentially contribute to most state-of-the-art, large-scale machine-learning problems. This talk is based on the paper [2303.03428].

About the Speaker

Dr. LIU Junyu (刘峻宇) is a theoretical physics postdoc scholar currently working for the University of Chicago and IBM. He is located in Uchicago in Prof. Liang Jiang's Group in the Pritzker School of Molecular Engineering. Dr. Liu graduated from School of the Gifted Young in the University of Science and Technology of China with a bachelor’s degree in physics, where his undergrad thesis was supervised by Prof. WANG Yi from HKUST. He then completed his PhD in the California Institute of Technology in physics in June 2021, with working experiences from the Walter Burke Institute for Theoretical Physics and the Institute for Quantum Information and Matter. Starting in 2022, he has been a co-founder of SeQure (a software company for blockchain security using AI and cryptography), a part-time research scientist of quantum software startup qBraid, and a scientific advisor of an AIGC startup Avolution. Dr. Liu is interested in theoretical physics and its relation to computation, including quantum computing, machine learning, data science and the commercial value of modern computing technologies.

About the Program

For more information, please refer to the program website at http://iasprogram.ust.hk/particle_theory.

For Attendees' Attention

• Seating is on a first come, first served basis.