Original Introduction to the topic
Generative artificial intelligence (genAI) like ChatGPT has to date mostly made its home in the massive data centers of service providers and enterprises. When companies want to use genAI services, they basically purchase access to an AI platform such as Microsoft 365 Copilot — the same as any other SaaS product.
One problem with a cloud-based system is that the underlying large language models (LLMs) running in data centers consume massive GPU cycles and electricity, not only to power applications but to train genAI models on big data and proprietary corporate data. There can also be issues with network connectivity. And, the genAI industry faces a scarcity of specialized processors needed to train and run LLMs. (It takes up to three years to launch a new silicon factory.)
“So, the question is, does the industry focus more attention on filling data centers with racks of GPU-based servers, or does it focus more on edge devices that can offload the processing needs?” said Jack Gold, principal analyst with business consultancy J. Gold Associates.
The answer, according to Gold and others, is to put genAI processing on edge devices. That’s why, over the next several years, silicon makers are turning their attention to PCs, tablets, smartphones, even cars, which will allow them to essentially offload processing from data centers — giving their genAI app makers a free ride as the user pays for the hardware and network connectivity.
– Companies traditionally purchase access to AI platforms in data centers, which can be costly and lead to network connectivity issues.
– The genAI industry faces a scarcity of specialized processors needed to train and run large language models (LLMs).
– Shifting genAI processing to edge devices such as PCs, tablets, smartphones, and cars is seen as the future to offload processing from data centers.
In conclusion, the future of genAI processing seems to be shifting towards edge devices rather than solely relying on data centers. This shift is driven by the need to reduce costs, address network connectivity issues, and tackle the scarcity of specialized processors required for genAI. As more silicon makers focus on producing dedicated devices for edge processing, the genAI industry is poised for a significant transformation.
Frequently asked questions
Q: Why is it important to shift genAI processing to edge devices?
A: Shifting genAI processing to edge devices is essential to reduce costs, address network connectivity issues, and tackle the scarcity of specialized processors required for genAI.
Q: What are some of the edge devices that genAI processing is being shifted to?
A: Silicon makers are turning their attention to PCs, tablets, smartphones, and even cars to offload genAI processing from data centers.
Q: What are the implications of the shift towards edge processing for genAI?
A: The shift towards edge processing for genAI signifies a transformation in the industry and a move to address the limitations of cloud-based systems.
In conclusion, the genAI industry is witnessing a significant shift towards edge processing, with a focus on utilizing PCs, tablets, smartphones, and other devices to offload the processing needs from data centers. This shift holds the promise of reducing costs, overcoming network connectivity issues, and addressing the scarcity of specialized processors required for genAI. As the industry continues to evolve, the role of edge devices in genAI processing is expected to play a crucial part in future developments.