AI Boom Gives Rise To 'GPU-as-a-Service' 22
An anonymous reader quotes a report from IEEE Spectrum: The surge of interest in AI is creating a massive demand for computing power. Around the world, companies are trying to keep up with the vast amount of GPUs needed to power more and more advanced AI models. While GPUs are not the only option for running an AI model, they have become the hardware of choice due to their ability to efficiently handle multiple operations simultaneously -- a critical feature when developing deep learning models. But not every AI startup has the capital to invest in the huge numbers of GPUs now required to run a cutting-edge model. For some, it's a better deal to outsource it. This has led to the rise of a new business: GPU-as-a-Service (GPUaaS). In recent years, companies like Hyperbolic, Kinesis, Runpod, and Vast.ai have sprouted up to remotely offer their clients the needed processing power.
[...] Studies have shown that more than half of the existing GPUs are not in use at any given time. Whether we're talking personal computers or colossal server farms, a lot of processing capacity is under-utilized. What Kinesis does is identify idle compute -- both for GPUs and CPUs -- in servers worldwide and compile them into a single computing source for companies to use. Kinesis partners with universities, data centers, companies, and individuals who are willing to sell their unused computing power. Through a special software installed on their servers, Kinesis detects idle processing units, preps them, and offers them to their clients for temporary use. [...] The biggest advantage of GPUaaS is economical. By removing the need to purchase and maintain the physical infrastructure, it allows companies to avoid investing in servers and IT management, and to instead put their resources toward improving their own deep learning, large language, and large vision models. It also lets customers pay for the exact amount of GPUs they use, saving the costs of the inevitable idle compute that would come with their own servers. The report notes that GPUaaS is growing in profitability. "In 2023, the industry's market size was valued at US $3.23 billion; in 2024, it grew to $4.31 billion," reports IEEE. "It's expected to rise to $49.84 billion by 2032."
[...] Studies have shown that more than half of the existing GPUs are not in use at any given time. Whether we're talking personal computers or colossal server farms, a lot of processing capacity is under-utilized. What Kinesis does is identify idle compute -- both for GPUs and CPUs -- in servers worldwide and compile them into a single computing source for companies to use. Kinesis partners with universities, data centers, companies, and individuals who are willing to sell their unused computing power. Through a special software installed on their servers, Kinesis detects idle processing units, preps them, and offers them to their clients for temporary use. [...] The biggest advantage of GPUaaS is economical. By removing the need to purchase and maintain the physical infrastructure, it allows companies to avoid investing in servers and IT management, and to instead put their resources toward improving their own deep learning, large language, and large vision models. It also lets customers pay for the exact amount of GPUs they use, saving the costs of the inevitable idle compute that would come with their own servers. The report notes that GPUaaS is growing in profitability. "In 2023, the industry's market size was valued at US $3.23 billion; in 2024, it grew to $4.31 billion," reports IEEE. "It's expected to rise to $49.84 billion by 2032."
Not unexpected (Score:4, Informative)
Given the low returns on "AI", you gotta do something with those graphic cards.
You may as well rent them out to the cryptobros for them to mine $TRUMP and $MELLANIA.
Otherwise the financial goals of "AGI" aren't going to happen, eh Sam?
Re: (Score:2)
How can I earn a few bucks by renting out a spare S3 Savage 3D?
Re: (Score:2)
How can I earn a few bucks by renting out a spare S3 Savage 3D?
Find some other people like you, build together a Beowulf cluster, plaster on an "API" and you're set.
Re: (Score:2)
"Find some other people like you, build together a Beowulf cluster..."
How many would "some" have to be to make it at least break even?
100? 1000? More?
Re: (Score:2)
Send me an email and I'll send you back a business plan for your investors and an invoice.
Re:Not unexpected (Score:4, Funny)
Re: (Score:3)
Welcome. Two conditions, you have to swear fealty and to be certain in your heart that Beowulf clusters of graphics are a thing.
Re: (Score:2)
Re: (Score:2)
Your hired :)
Re: (Score:3)
Pretty sure you can't mine TRUMP. It's a memecoin. Most of them aren't mineable. DOGE was an exception since it has its own chain (being a fork of Litecoin).
That being said, you can rent rigs for mining. You've been able to do so for years now.
Re: (Score:2)
Could be. The point is that "AI" rigs are available because there is less need for those "AI" services than what was projected by the aibros.
Hardware as a service (Score:2)
GPU-as-a-service is just a specialized hardware as a service, which is something that looks convenient to corporate management the same way as consultants looks convenient to corporations but in the end means higher costs.
Someone has sold the idea of "you own nothing and you'll be valuable" to corporate management.
Article link is wrong (Score:3)
Re: (Score:1)
That's why OpenAI had to make a devil's bargain with Azure, they're desperate for cheap compute.
Old As The Hills (Score:2)
ASIC's will come (Score:2)
ASIC's will come in time.
Basically we've seen the peak already with the 3090, that has been no real gain from the 4090 or 5090 that we're seeing. All nvidia has done is packed more in to the die, and tried to get as close to the 600w limit, and when the last card was also pretty close to 600w, kinda hard to get any more power out of the desktop GPU now.
So now is the time for fixed-logic ASIC's for resusable logic, going back 20 years back to when hardware T&L was a buzzword. GPU's are highly programable
Re: (Score:2)
ASIC's will come in time.
Basically we've seen the peak already with the 3090, that has been no real gain from the 4090 or 5090 that we're seeing.
Uhh, no one is doing serious AI with their gaming GPUs.
ASICs might one day encroach on Nvidia's market dominance. It's not clear when and if that will happen. Remember that the most mature ASIC is Google's TPU, which is already on its 6th iteration and at least 10th year of development. Maybe the 11th year will be the magical year, maybe the 20th year, maybe never. We'll see.
It's also not clear if a viable Nvidia competitor will appear as an ASIC or a GPU. My bet is on an AMD/Intel GPU over an ASIC, al
Re: (Score:2)
AI is not a single faceted mathematical problem. we already have specific purpose build devices for AI workloads, they are referred to as GPUs because they follow a similar kind of generalised architecture to consumer stuff, but really they are nothing alike. You can't throw ASICs at every problem, ASICs are ideal for problems with highly defined mathematical boundaries. Training AI models is not one of them, there are multiple steps in the model training that involve a variety of mathematical problems, whi
Re: (Score:2)
Also before you point to "AI acceleration" in consumer products note that the application of an AI model is a different problem than training a model.
Re: (Score:2)
I'd call it a multicore processor but they call it an ASIC presumably because they designed it for a specific type of workload. So then, is a H200 a GPU or is it an ASIC? It is designed to run CUDA workloads well isn't it? Can it run ROCm? If it isn't GPU "but really they are nothing alike" then it sounds like an ASIC