Isaac Liao

I'm a Master's student advised by Max Tegmark in the Tegmark AI Safety Group at MIT, doing research on mechanistic interpretability (machine learning). For my undergrad at MIT, I double majored in Computer Science and Physics, and did research on meta-learned optimization in the lab of Marin Soljačić. Within machine learning, my interests include the minimum description length, variational inference, hypernetworks, meta-learning, optimization, and sparsity.

In my leisure time, I enjoy skating, game AI programming, and music. I won the Battlecode AI Programming Competition in 2022. I was a silver medalist in the International Physics Olympiad (IPhO) in 2019 and an honorable mention in IPhO 2018.

Email  /  CV  /  Resume  /  Google Scholar

Machine Learning Research

Compilation of over 30 problems and corresponding datasets for learning algorithmic tasks on sequences. Novel methods for conversion of learned recurrent neural networks into interpretable python code equivalents, for model verification.

Introduction of a novel hypernetwork architecture for generative modeling of neural network weights. Architecture is a merge of Pareto hypernetworks, hierarchical VAEs, and graph transformers. Reverse-engineering of learned algorithms, with algorithmic phase transitions using order parameters and visualization via force-directed graph drawing.

Introduces a novel machine learning optimization algorithm which blends learn to optimize (L2O) meta-learning techniques with quasi-Newton optimization methods using sparse neural networks. Theoretical results regarding convex and nonconvex stochastic convergence and sparse neural network expressiveness, with experimental support.

Research-like Class Projects

Targeted movie recommendation systems using the Low-Rank Approximation with Alternating Least Squares method of large matrix completion. Reinterpretation of the algorithm in the Bayesian formulation, and an extension of the resulting Bayesian model to allow for uncertainty in user preferences and movie characteristics. Experiments to show that this improves recommendation accuracy.

Exploration of the expressiveness properties of sparse neural network architectures. Extensions of theorems about the ability of sparse architectures to replicate the behavior of any other sparse architecture. Experimental evaluation of the ability of sparse architectures to perform compositionally sparse linear operations.

Treatment of variational inference from the point of view of information compression, with application to images. Ideation, refinement, theoretical analysis, and empirical testing of ELBO maximization-based lossless image compression schemes resembling VAEs and BNNs. Reparameterization trick, hierarchical depth, and KL annealing schedules, and rejection sampling all independently reinvented without prior knowledge of existing variational inference techniques.

Rederivation of joint eigenvalue distribution of random Hermitian matrices, using a combination of second-order quantum perturbation theory, Metropolis-Hastings, and Brownian motion. Rederivation of Wigner semicircle law. Connections to chaotic quantum billiards and application to emission spectra of quantum dots.

Education Research

Novel methods for applying large language models to assist in teaching physics to ~700 students. Large language models for answering student questions and generating physics problems used for teaching. Drafting of manuscripts for publication in journals on physics education and AI in education. Course website maintenance. Office hours, grading, exam proctoring.

Miscellaneous

Summary report for my strategy in the Battlecode 2021 Game AI programming competition, in which I got 7th place worldwide.

This website was produced from a template made by Jon Barron.