Pinned
Ever looked at the attention operation and said "hang on, that's a one-point function!"?
Vasu Shyam
632 posts
Vasu Shyam
@vasud3vshyam
Currently working as a machine learning researcher at a Silicon Valley startup. Former physics postdoc at Stanford and Branco Weiss fellow.
San Francisco CA
Joined January 2023
- Replying to @vasud3vshyamWell, if the answer to any of these questions was "no" then consider reading: arxiv.org/pdf/2408.04093 that I co authored with @J_Pilault @nshepperd1, @BerenMillidge, @QuentinAnthon15Zyphra is proud to release Tree Attention, a fast inference method for extremely large sequence lengths • 8x faster inference speed vs. Ring Attention • 2x less peak memory • low data communication volumes Paper: arxiv.org/abs/2408.04093 Code: github.com/Zyphra/tree_at… A 🧵
- Replying to @vasud3vshyamHaving noticed that, did you then write down the generating function?
- Replying to @vasud3vshyamAnd then did you see how much faster than Ring Attention this method ends up being for decoding?
- Replying to @vasud3vshyamThen, did you happen to recall that thanks to automatic differentiation (timvieira.github.io/blog/post/2016…) the time complexity to compute the gradient of a function is roughly equivalent to the complexity of computing the function itself?
- Finally managed retire early from my professional physics research career (I know, I know, in time it would have retired me). Eagerly looking forward to going full crackpot as an amateur.
- Replying to @vasud3vshyamConsequently, did you realize that the efficient tree reductions here can be done on a DGX cluster via NCCL Allreduce in a topology-aware, efficiently overlapped manner?
- Replying to @vasud3vshyamIf you thought this was interesting, wait till you see what else my incredible colleagues at @ZyphraAI are up to!
- Replying to @vasud3vshyamOK that was a fun 15 minutes of twitter fame. Now for the downfall - the tweet at the top's got an embarrassing typo, bonus points for whoever catches it
- John Donoghue @JFdonoghue1033 kindly joined my podcast yesterday and clearly explained in what sense we already have quantum gravity at low energies:
- Replying to @ylecunThanks for sharing! Another little trick that might amuse you is that we identified a function which upon minimization produces the forward pass of the attention block:
- Replying to @CburgesCliffThanks for this, Cliff. What's a good reference that you'd recommend to learn about the current status of the Baryon Asymmetry problem?
