Nvidia to license technology from inference chip startup Groq in reported $20B deal

Artificial intelligence chip startup Groq Inc. today announced that Nvidia Corp. will license its technology on a nonexclusive basis.

The deal will also see the graphics card maker hire several key Groq employees. They include founding Chief Executive Officer Jonathan Ross and president Sunny Madra. Alex Davis, the CEO of Groq investor Disruptive Technology Advisers, told CNBC that the transaction is worth $20 billion.

Transactions through which a company gains access to a startup’s talent and technology without buying it outright are known as reverse acquihires. They avoid the antitrust scrutiny often associated with acquisitions. Over the past three years, tech giants such as Microsoft Corp. and Meta Platforms Inc. have inked multiple such deals to advance their AI roadmaps.

The $20 billion that Nvidia will reportedly pay to license Groq’s intellectual property is a $13.1 billion premium to the startup’s September valuation. The companies states that the license covers Groq’s “inference technology,” which is presumably a reference to its flagship LPU inference chip.

The company claims that the LPU can run inference workloads using ten times less power than graphics cards. According to Groq, the chip’s efficiency stems from the fact that it features a deterministic design. That means it can control the timing of calculations with a high degree of precision. In contrast, standard, nondeterministic chips often experience unexpected processing delays that mix up the order in which computations are completed.

The LPU also optimizes AI performance in other ways. It includes several hundred megabytes of on-chip SRAM, the fastest type of memory on the market. That memory pool is more performant than the HBM memory used by graphics cards and requires less power.

Groq links together LPU-equipped servers into inference clusters using an internally developed interconnect called RealScale. According to the company, the technology addresses a technical issue known as crystal-based drift that makes it difficult to coordinate AI servers.

Processors use a clock to control the frequency at which their circuits carry out calculations. Usually, the clock is implemented as a tiny quartz crystal. Crystal-based drift is a phenomenon that causes the clock to unexpectedly slow its frequency, which can introduce inefficiencies into AI inference workflows. Groq says that RealScale can automatically adjust processor clocks to mitigate the issue.

Nvidia CEO Officer Jensen Huang reportedly informed employees in an internal memo that “we plan to integrate Groq’s low-latency processors into the Nvidia AI factory architecture, extending the platform to serve an even broader range of AI inference and real-time workloads.”

Groq will continue operating as an independent company following the transaction. Simon Edwards, the company’s chief financial officer, will succeed Jonathan Ross as CEO.

As of July, Groq reportedly expected to end the year with $500 million in revenue. The company sells access to its chips through a cloud platform called GroqCloud. The platform also provides a library of open-source AI models and tools such as search engines that those models can use to process prompts. 

Image: Groq