Ziming Liu (@ZimingLiu11) / X

Ziming Liu

656 posts

Ziming Liu

@ZimingLiu11

Assistant Professor @ Tsinghua CollegeAI, Postdoc @ Stanford, PhD @ MIT, BS @ PKU. Physics of AI, interpretability, Structuralism, KAN

kindxiaoming.github.io

Joined May 2021

Pinned
Ziming Liu
@ZimingLiu11
May 1, 2024
MLPs are so foundational, but are there alternatives? MLPs place activation functions on neurons, but can we instead place (learnable) activation functions on weights? Yes, we KAN! We propose Kolmogorov-Arnold Networks (KAN), which are more accurate and interpretable than MLPs.🧵
GIF
1M
Ziming Liu
@ZimingLiu11
May 5, 2023
To make neural networks as modular as brains, We propose brain-inspired modular training, resulting in modular and interpretable networks! The ability to directly see modules with naked eyes can facilitate mechanistic interpretability. It’s nice to see how a “brain” grows in NN!
00:00
417K
Ziming Liu
@ZimingLiu11
May 17, 2025
Interested in the science of language models but tired of neural scaling laws? Here's a new perspective: our new paper presents neural thermodynamic laws -- thermodynamic concepts and laws naturally emerge in language model training! AI is naturAl, not Artificial, after all.
113K
Ziming Liu
@ZimingLiu11
Sep 23, 2022
Diffusion models arise from thermodynamics, but physics can definitely do more for generative models! Inspired by electrostatics, we propose Poisson Flow Generative Models (PFGM) with SOTA performance. Joint work with @xuyilun2 @tegmark and Tommi Jaakkola. arxiv.org/abs/2209.11178
Ziming Liu
@ZimingLiu11
Aug 18, 2023
Deep learning has many mysterious phenomena, and grokking is one of the extreme. Want to catch up with the grokking literature? I've compiled a one-page summary of what's going on in the grokking world. Enjoy! :-) kindxiaoming.github.io/pdfs/grokking_…
103K
Ziming Liu
@ZimingLiu11
May 3, 2024
Thanks everyone for cheering applause and constructive criticism. I wrote a few paragraphs responding to the recent KAN hype. In short, I think it is too early to say KANs will replace MLPs, but there are indeed many interesting directions to explore.
pykan/README.md at master · KindXiaoming/pykan
From github.com
64K
Ziming Liu
@ZimingLiu11
Oct 12, 2023
Mechanistic interpretability is not only for ML or LLM, but is even more promising for Science! A few months ago, we proposed a brain-inspired method (BIMT) for NN interpretability; Now we're happy to see that it can give something back to neuroscience - Growing brains in RNNs!
115K
Ziming Liu
@ZimingLiu11
Jan 22, 2025
New paper🚨: Physics of Skill Learning Training dynamics is complicated, but are there simple "physical laws" behind it? We take physicists' approach of simplification and abstraction: Simple models like "spherical cows" are surprisingly effective! arxiv.org/pdf/2501.12391 🧵
58K
Ziming Liu
@ZimingLiu11
May 19, 2023
We now release BIMT codes and welcome you to train your own modular & interpretable networks, like growing a "brain"! This thread 🧵will cover the basic idea & beyond (1/N) 📃Paper: arxiv.org/abs/2305.08746 📷Code: github.com/KindXiaoming/B… 🔗Demo:colab.research.google.com/drive/1hggc5Ta…
00:00
79K
Ziming Liu
@ZimingLiu11
Oct 4, 2023
Extremely boring paper alert‼️ Training 100000 small networks, studying their statistics and interpreting them. This "boring" stuff leads to an intriguing mechanism for neural scaling laws, which happens to explain the 0.34 exponent observed in Chinchilla (our prediction is 1/3).
75K
Ziming Liu
@ZimingLiu11
Oct 3, 2024
Why are KANs good at function fitting? Happy to share our theoretical work on understanding the approximation power and spectral biases of KANs! Compared with MLPs, KANs are O(G) more parameter efficient, and suffer much less from spectral biases! KANs are not MLPs, after all! 🧐
39K
Ziming Liu
@ZimingLiu11
May 26, 2023
Many scientific problems hinge on finding interpretable formulas that fit data, but neural networks are the outright opposite! Check out our recent work that make neural networks modular and interpretable. If you have interesting datasets at hand, we're happy to collaborate!
00:00
62K
Ziming Liu
@ZimingLiu11
May 3, 2024
I apologize for not explaining B-splines very well in the KAN paper, will aim to make the definitions more explicit in an updated version. Here’s a really nice writeup on the potential research questions for KANs, including key technicalities used in the paper.
Simone Scardapane
@s_scardapane
May 2, 2024
*Kolmogorov-Arnold Networks (KANs)* by @ZimingLiu11 et al. Since everyone is talking about KANs, I wrote some notes on Notion with a few research questions I find interesting. First time I do something like this, give me some feedback. 🙃 sscardapane.notion.site/Kolmogorov-Arn…
56K
Ziming Liu
@ZimingLiu11
Aug 20, 2024
Excited to share our new paper KAN 2.0: Kolmogorov-Arnold Networks meet Science 🚀 The problem with AI + Science is that these two disciplines use different "languages" (connectionism vs symbolism). KAN 2.0 attempts to unify them: smooth transitions from science to KAN and back.
48K