Nvidia GPUs are the water that feeds today's flourishing AI ecosystem. Without the power that the A100 and H100 chips provide, ChatGPT and most other new generative AI services wouldn't exist.
With the boom in AI startups and services, there's so much demand for these crucial components that Nvidia, and manufacturing partner TSMC, can't make enough of them. While the demand has been great for Nvidia, which gained almost $200 billion in market cap in a single day last month after announcing higher revenue guidance for its GPU chips, it's been less fortunate for AI startups. Prices have soared and shortages are common.
This is giving Big Tech companies another huge advantage over smaller upstarts. If you really want to compete in this new AI world, you have to train your own models with your own data, which can require a large amount of GPUs. Otherwise, you will just be another app on someone else's platform.
Microsoft, and its partner OpenAI, knows this, Google does too. Even Adobe does. They are rushing to train large foundation models on huge amounts of data and have the benefit of billions of dollars to invest in this expensive process.
Many startups can't afford that, or they just can't get hold of the chips. Even tech juggernaut Microsoft is facing a hardware crunch, going as far as to ration internal access to GPUs to save processing power for its AI-powered Bing chatbot and AI Microsoft Office tools, The Information reported.
So, some venture capitalists are taking unusual steps to help.
Nat Friedman, the former CEO of GitHub, and Daniel Gross, who has backed GitHub, Uber, and a host of other successful startups, have purchased thousands of GPUs and set up their own AI cloud service.
Called Andromeda Cluster, the system has 2,512 H100 GPUs and is capable of training a 65 billion parameter AI model in about 10 days, the VCs said. That's not the largest model out there, but it's a sizeable one.
The catch: This is only available for startups backed by Friedman and Gross. Still, the move is being praised already.
"Individual investors doing more to support compute-intensive startups than most governments. (Very cool project!)," tweeted Jack Clark, a cofounder of AI startup Anthropic.
