Nvidia has taken a big step in AI hardware by licensing technology from Groq, a chip designer specialized in AI inference. While Nvidia hasn’t bought Groq outright, it has secured a non-exclusive license to use Groq’s intellectual property and hired some of its top engineers. This move hints at Nvidia’s strategy to expand its AI chip offerings without full acquisition.

What Groq Does and Why It Matters

Groq makes chips called language processing units (LPUs), optimized for AI inference tasks. These chips are different from Nvidia’s main GPUs, which are mainly used to train AI models. Groq’s LPUs are designed to be lower-cost and consume less power, making them suitable for deploying AI in real-world applications where efficiency matters.

Beyond selling chips, Groq also rents out its hardware through a service called GroqCloud, providing inference-as-a-service. The company announced the licensing deal on December 24, stating that key leaders like founder Jonathan Ross and President Sunny Madra would join Nvidia to help develop and scale Groq’s technology. The deal could be worth up to $20 billion, according to reports.

Strategic Moves in the AI Chip Market

This partnership comes at a time when the AI chip market faces supply chain issues. Nvidia’s CFO recently highlighted that some of their chips are fully sold out due to high demand. A major challenge is the shortage of high-bandwidth memory, which is critical for AI workloads.

Groq’s chips use static RAM (SRAM), which is faster and uses less power than the dynamic RAM (DRAM) typically used in Nvidia’s high-end GPUs. Since SRAM is less scarce and more affordable, licensing Groq’s technology could help Nvidia diversify its memory sources and reduce reliance on expensive, hard-to-get memory chips.

By licensing Groq’s tech instead of buying the company, Nvidia avoids taking over GroqCloud’s entire business. This is important because Nvidia has been shifting its cloud services, such as DGX Cloud, towards internal engineering support rather than standalone services. The licensing deal also helps Nvidia stay flexible and avoid some regulatory hurdles that come with acquisitions.

Overall, this move shows Nvidia’s focus on expanding its AI hardware capabilities by leveraging innovative technologies and talent from smaller players. It’s a strategic way to stay ahead in a fast-growing and competitive AI market without overextending its resources.

Inspired by

• https://www.infoworld.com/article/4112134/nvidia-licenses-groqs-inferencing-chip-tech-and-hires-its-leaders-2.html

Sources

• networkworld.com
• groq.com
• techcrunch.com
• networkworld.com
• theinformation.com