Projects

Full List (Chronological)

Improved Mean Flows Highlight Compute Transparency Champion
Zhengyang Geng*, Yiyang Lu*, Zongze Wu, Eli Shechtman, J. Zico Kolter, and Kaiming He
TL;DR: Diagnosing and improving failure modes of fast one-step generative models.
[Paper] [Code]
One-step Latent-free Image Generation with Pixel Mean Flows
Yiyang Lu*, Susie Lu*, Qiao Sun*, Hanhong Zhao*, Zhicheng Jiang, Xianbang Wang,
Tianhong Li, Zhengyang Geng, and Kaiming He
TL;DR: One-step latent-free image generation directly in pixel space.
[Paper] [Code]
Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning
Benhao Huang, Zhengyang Geng^†, and J. Zico Kolter
TL;DR: Task-conditioned attractors enable scalable latent reasoning.
[Paper] [Code]

Mean Flows for One-step Generative Modeling
Zhengyang Geng, Mingyang Deng, Xingjian Bai, J. Zico Kolter, and Kaiming He
TL;DR: Learning to solve generative dynamics at training time.
[Paper] [JAX Code] [PyTorch Code]
Consistency Models Made Easy
Zhengyang Geng, William Luo, Ashwini Pokle, and J. Zico Kolter
TL;DR: Easy Consistency Tuning through Self Teacher.
[Paper] [Blog] [Code] [BibTeX]
Diff-Instruct*: Towards Human-Preferred One-step Text-to-image Generative Models
Weijian Luo, Colin Zhang, Debing Zhang, and Zhengyang Geng
TL;DR: Score-based preference alignment for one-step text-to-image models.
[Paper] [Code]

One-Step Diffusion Distillation through Score Implicit Matching
Weijian Luo, Zemin Huang, Zhengyang Geng, J. Zico Kolter, and Guo-jun Qi
TL;DR: Data-free one-step diffusion distillation via score implicit matching.
[Paper] [Code]
Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
Tianle Cai*, Yuhong Li*, Zhengyang Geng, Hongwu Peng, and Tri Dao
TL;DR: Simple LLM acceleration with multiple decoding heads and self-verification.
[Paper] [Blog] [Code]

TorchDEQ: A Library for Deep Equilibrium Models
Zhengyang Geng and J. Zico Kolter
TL;DR: Modern fixed-point systems in PyTorch.
[Report] [Code] [Colab Tutorial] [Doc] [DEQ Zoo]
1-Step Diffusion Distillation via Deep Equilibrium Models
Zhengyang Geng*, Ashwini Pokle*, and J. Zico Kolter
TL;DR: Generative Equilibrium Transformer (GET) as a strong 1-step diffusion learner.
[Paper] [Code] [BibTeX]
Equilibrium Image Denoising With Implicit Differentiation
Qi Chen, Yifei Wang, Zhengyang Geng, Yisen Wang, Jiansheng Yang, and Zhouchen Lin
TL;DR: Equilibrium image denoising with implicit differentiation.
[Paper] [BibTeX]

Deep Equilibrium Approaches To Diffusion Models
Ashwini Pokle, Zhengyang Geng, and J. Zico Kolter
TL;DR: Parallel diffusion decoding via fixed-point equations.
[Paper] [Code] [BibTeX]
Eliminating Gradient Conflict in Reference-based Line-art Colorization
Zekun Li, Zhengyang Geng, Zhao Kang, Wenyu Chen, and Yibo Yang
TL;DR: Investigating and alleviating gradient conflicts in attention training.
[Paper] [Code] [BibTeX]
Deep Equilibrium Optical Flow Estimation
Shaojie Bai*, Zhengyang Geng*, Yash Savani, and J. Zico Kolter
TL;DR: Harder problems, more compute, better convergence and performance.
[Paper] [Code] [BibTeX]

On Training Implicit Models
Zhengyang Geng*, Xin-Yu Zhang*, Shaojie Bai, Yisen Wang, and Zhouchen Lin
TL;DR: Inexact gradient training can be cheap, fast, and stable.
[Paper] [Slides] [Poster] [Code] [BibTeX]
Residual Relaxation for Multi-view Representation Learning
Yifei Wang, Zhengyang Geng, Feng Jiang, Chuming Li, Yisen Wang, Jiansheng Yang, and Zhouchen Lin
TL;DR: Equivariant contrastive learning replaces invariant contrastive learning.
[Paper] [Slides] [BibTeX]
Is Attention Better Than Matrix Decomposition?
Zhengyang Geng*, Meng-Hao Guo*, Hongxu Chen, Xia Li, Ke Wei, and Zhouchen Lin
TL;DR: Optimization (matrix decomposition) as attention.
[Paper] [Code] [Blog Series 1 (zh), 2 (zh), 3 (zh)] [Poster] [BibTeX]