Indian startup Ziroh Labs, in collaboration with researchers at IIT Madras, has designed an affordable system that it says can run large artificial intelligence (AI) models without requiring advanced computing chips from the likes of Nvidia Corp.
According to a report by Bloomberg, the company’s framework is called Kompact AI and was developed in partnership with the Indian Institute of Technology, Madras.
Ziroh Labs has claimed that the platform enables AI to run on the central processing units (CPUs) found in everyday computing devices as opposed to the coveted, and costly, graphics processing units (GPUs) that have been the linchpin of the artificial intelligence boom.
Ziroh Labs said it can optimise leading AI models to run on personal computers. In a demonstration event this week, the team of researchers showed their product working on a laptop that uses a shelf-bought Intel Xeon processor and querying models such as Meta’s Llama 2 and Alibaba’s Qwen 2.5.
While other firms have also used CPUs to handle some inference workloads. Ziroh Labs said its approach leads to high-quality results. The startup also said that its technology has been tested by US chipmakers Intel and Advanced Micro Devices.
Why is the use of CPUs in AI models a landmark?
A growing number of AI developers have touted efficiency gains that let them use fewer chips in the months following the viral success of China’s DeepSeek, which purportedly built a competitive AI model for a fraction of the cost of its US peers. Ziroh Labs’ approach primarily focuses on the process of inference, or operating AI systems after they’ve been trained.
{{/usCountry}}A growing number of AI developers have touted efficiency gains that let them use fewer chips in the months following the viral success of China’s DeepSeek, which purportedly built a competitive AI model for a fraction of the cost of its US peers. Ziroh Labs’ approach primarily focuses on the process of inference, or operating AI systems after they’ve been trained.
{{/usCountry}}“It’s going to have a very profound market impact in the years ahead,” Bloomberg quoted William Raduchel, former chief strategy officer of Sun Microsystems.
Raduchel is a tech adviser to the startup who spoke virtually at the event.
As in other countries, developers in India have struggled to afford and get access to top-of-the-line Nvidia chips to help build and support AI products. The shortage of GPUs risks hindering the speed and scale of local AI research and deployment.
“The AI divide is because only those with high-end, expensive GPU-powered resources can access, develop, and deploy powerful AI,” said V. Kamakoti, director at the Indian Institute of Technology, Madras.
“We’re demonstrating that you don’t need a revolver to kill a mosquito,” he added.