Pinned
Modal
1,510 posts
Modal
@modal
AI infrastructure that developers love 💚
Run inference, sandboxes, batch processing, training, and many other things on Modal
Replying to @modal @jianchen1799 and @liin1211You can find the drafter on @huggingface, where we've each released an identical copy of the weights. Kinda like getting matching tats with your bestie Our copy is here: huggingface.co/modal-labs/Qwe… The repos include scripts that reproduce our benchmark showing superiority over MTP:
- We worked with @lmsysorg and z-lab.ai to - integrate DFlash spec into @sgl_project - make it faster with overlap - train a DFlash drafter for @Alibaba_Qwen 397B-A17B The result: up to 4.3x greater throughput over baseline and 1.5x over native MTP.
- Modal reposted
williamsburg too 🗽
00:00
around sf 🌉 - Modal reposted
Last fall, we shared our deep dive on FA4 internals. But we didn't stop at grokking the kernel. Since then, we've been developing improvements for inference performance and upstreaming them. This blog post explains those contributions. modal.com/blog/flash-att…
- Modal repostedTried to squeeze the most important bits about the entire stack for cloud deployment of transformer inference, from application layer concerns to hardware, debugging, and o11y, into one talk. Had to operate at a very high tok/s! youtube.com/watch?v=ZUdIsR…
- With today's launch of Nemotron 3 Ultra, @nvidia continues to expand its investment in open-source AI. Their flagship frontier-reasoning model, built for long-running autonomous agents, is available Day 0 on Modal. - 550B with 55B active parameters - Hybrid Transformer-Mamba MoE
- Modal repostedwe're hosting some parties to celebrate our C 💚 exclusive swag at both ofc
00:00
00:15We're bringing together our friends and community to celebrate our Series C. Join us at Noguchi's Sunken Garden in NYC on June 16th or at the Legion of Honor in SF on June 25th. Invites are limited, apply here: modal.com/c-function - Modal reposted
I’m so excited about the launch of ESMFold2, ESMC, and the new ESM Atlas. This was a massive team effort, and I’m grateful to have worked with such an incredible group @biohub. A headline result I’m especially excited about: ESMFold2 can design minibinders and antibodies with - We're bringing together our friends and community to celebrate our Series C. Join us at Noguchi's Sunken Garden in NYC on June 16th or at the Legion of Honor in SF on June 25th. Invites are limited, apply here: modal.com/c-function00:00
