OpenAI says its unhappy with Nvidia inference hardware, now looking at AMD, Cerebras, Groq

OpenAI isn’t happy with Nvidia’s AI chips anymore, especially when it comes to how fast they can answer users. The company started looking for other options last year, and now it’s talking to AMD, Cerebras, and was even talking to Groq before that got shut down.

This tension started getting real when OpenAI realized Nvidia’s chips weren’t fast enough for specific things like writing code and handling software-to-software tasks.

One insider allegedly said OpenAI wants new chips to handle at least 10% of its inference needs going forward. That’s the part where the AI replies to users, not the part where it learns stuff.

OpenAI wants faster chips for coding and user replies

Most of OpenAI’s current work still runs on Nvidia, but behind the scenes, it’s testing chips that could make everything faster.

This includes chips packed with SRAM, which helps speed things up by putting memory right next to the processor. Nvidia and AMD still use memory that sits outside the chip, which slows things down.

People inside OpenAI pointed to Codex, the tool that writes code, as the place where the slowness was the biggest problem. Some staff even blamed the weak performance on Nvidia’s hardware. In a press call on January 30, OpenAI CEO Sam Altman said, “Customers using our coding models will put a big premium on speed for coding work.”

Sam added that regular ChatGPT users don’t care about speed as much, but for developers and companies, every second counts. He said OpenAI had just signed a deal with Cerebras to help speed things up.

At the same time, companies like Anthropic and Google are getting better results using their own chips. Google’s TPUs are built specifically for the kind of work inference needs. That’s made them faster at responding, especially for models like Claude and Gemini.

OpenAI-Groq talks shut down after Nvidia license deal

OpenAI was also in talks with Groq, another startup building fast chips, but those conversations didn’t go far. Nvidia came in and signed a $20 billion licensing deal with Groq. That gave Nvidia access to Groq’s designs and killed OpenAI’s plans to work with them.

A source close to the situation said Groq’s chips were built exactly for what OpenAI needed. But once Nvidia locked in the deal, that door closed. Even though the license was non-exclusive, Groq is now focusing on cloud-based software, and Nvidia took some of Groq’s chip designers for itself.

Cerebras, on the other hand, said no when Nvidia tried to buy them. Instead, they went ahead and made their own deal with OpenAI. Groq also got investment offers putting its value around $14 billion, but that’s now shifted since it’s tied up with Nvidia.

OpenAI hasn’t walked away from Nvidia completely. In a public statement, a spokesperson said, “We rely on Nvidia to power the vast majority of our inference fleet,” and called their performance per dollar the best in the market. Nvidia also said, “Customers continue to choose Nvidia for inference because we deliver the best performance and total cost of ownership at scale.”

$100 billion Nvidia investment deal still stuck in limbo

Last year, Nvidia said it planned to invest up to $100 billion into OpenAI. That cash was meant to help OpenAI buy more advanced chips, and in return, Nvidia would get a stake in the company. Reuters said the deal was supposed to close in a few weeks. It still hasn’t.

While that deal stalled, OpenAI went ahead and signed agreements with AMD and others to test chips that could compete directly with Nvidia’s. But as OpenAI changed its product plans, the kind of hardware it needed also changed. That slowed the talks even more, someone familiar with the situation said.

On Saturday, Nvidia CEO Jensen Huang was asked about the friction. He said, “That’s nonsense,” and insisted Nvidia still plans to invest big in OpenAI. But behind the scenes, it’s clear both sides are exploring their options.

At the same time, Nvidia has been shopping for new chip ideas. It reached out to both Cerebras and Groq to see if they’d be open to getting bought. Cerebras turned that down and doubled down on its deal with OpenAI.

Right now, OpenAI is using GPT4o to power most of its services. But the way things are going, at least some of that work will run on chips from AMD or Cerebras in the near future. The company isn’t trying to ditch Nvidia completely, but it’s clear it wants more control over how fast its systems work.

Sharpen your strategy with mentorship + daily ideas - 30 days free access to our trading program