Neural Notes: Big tech is coming for Nvidia’s AI chippies

Welcome back to Neural Notes, a column where I explore the biggest — and sometimes the weirdest — AI news of the week. In this edition: it seems like everyone wants some of that Nvidia chipset money, with Apple, Intel, Google and Meta unveiling their latest offerings within days of each other.

This week’s flurry of announcements reflects a significant shift in the AI landscape (particularly generative) which has been largely focused on the software side.

Now major tech players are seeking to carve out their own niches in the lucrative AI hardware market. While this domain is being dominated by Nvidia, recent reports have shown that the rising AI tide is raising, well, a lot of boats.

As companies vie for a piece of Nvidia’s $338 billion pie, the broader impact on the industry will likely include accelerated innovation, more tailored AI solutions across various sectors, and possibly a shift in who holds the technological high ground in the next five to ten years.

With that in mind, let’s take a peek at the chip-shaped wares big tech are peddling.

Google takes up arms

Google is back in the AI-adjacent headlines after considering charging for search results powered by artificial intelligence just last week.

During the annual Google Cloud Next conference in Las Vegas, Google announced the release of its Cloud TPU v5p, which it claims can train large language models nearly three times faster than its predecessor.

“Now in their fifth generation, these advancements [to Google’s TPUs] have helped customers train and serve cutting-edge language models,” Google CEO, Sundar Pichai, said.

Google also introduced its first Arm-based CPU, the Google Axion. This is a big deal, with Arm CPUs being known to not only offer a lower price point, but lower power requirements.

Google also took a swing at other Arm-computing competitors in the market like Apple, Intel and Nvidia. According to Google Cloud CEO, Thomas Kurian, the Axion offers “up to 30% better performance than the fastest general-purpose Arm-based instances available in the cloud today” and “up to 50% better performance and up to 60% better energy efficiency”.

The move to Arm-based CPUs allows Google to potentially increase the energy efficiency of its data centers, a growing concern across the industry given the environmental impact of large-scale computing operations. The introduction of Axion signifies Google’s attempt to diversify its processor technology and reduce its reliance on traditional x86 architecture, which dominates the CPU market.

By adopting Arm’s infrastructure, Google aims to enhance the integration of AI capabilities into its cloud services, offering a platform that they claim will allow for greater customisation and optimisation of power consumption and operational efficiency.

This shift seems to reflect Google’s broader strategy to tailor its hardware solutions more closely to its operational needs and the requirements of its cloud customers.

Intel’s Gaudi approach to AI

Intel has unveiled its Gaudi 3 AI accelerator chip at the Intel Vision event, which it claims surpasses Nvidia’s H100 in training large language models and competes strongly against the H200 in inferencing performance.

Jeni Barovian, general manager of AI Solutions Strategy and Product Management at Intel, emphasised the company’s commitment to providing customers with more choices and better value.

“Our customers are looking for alternatives to Nvidia’s powerful and popular chips, and we are building a comprehensive AI strategy with open, scalable systems that will be attractive for enterprises across all AI segments,” Barovian said.

Intel reports that the Gaudi 3 doubles the AI compute performance using the 8-bit floating point format and quadruples it with the 16-bit BFLOAT16 format compared to its previous iteration, Gaudi 2.

Intel plans to roll out both air-cooled and liquid-cooled versions of the Gaudi 3, with samples already being distributed to customers. The wider availability of these chips is intended to bolster Intel’s position in the AI hardware market, directly challenging Nvidia’s dominance and offering a shiny alternative for cloud service providers and enterprises aiming to scale their AI operations.

Gaudi indeed.

Apple tries to recover declining sales with boosted AI offering

Apple is set to refresh its entire lineup of Macs with the next-generation M4 chips, highlighting a significant push to enhance AI capabilities in its devices.

Bloomberg‘s Apple rumour guru Mark Gurman reports that these new chips, following the recent M3 series, will focus more intensively on AI performance, addressing the perceived lag in Apple’s AI readiness compared to competitors like Microsoft, which has also been advancing in AI with new Qualcomm Snapdragon X Series chips.

Apple’s new M4 chips will be introduced in three variants: Donan, Brava, and Hidra, each tailored to different levels of performance needs but all designed to substantially outperform the current M3 chips in AI processing tasks. This development comes as Apple aims to recover from a 27% drop in Mac sales over the last fiscal year, pushing innovation to regain its edge.

In preparation for these launches, Apple is also reportedly planning enhancements to macOS, integrating it more deeply with the capabilities of the M4 chips. This implies that Apple’s focus is not only on raw performance but maximising the efficiency and effectiveness of AI applications directly within its ecosystem.

We’re likely to see significant details about the M4 chips at Apple’s World Wide Developer Conference in June.

Zuck would also like some AI bucks

Coming up the rear we have Meta and the release of its next-gen Meta Training and Inference Accelerator (MTIA).

The social media claims the MTIA delivers up to three times better performance compared to its predecessor for specific AI tasks such as ad ranking and recommendations on its platforms. Meta describes this performance improvement as stemming from controlling the entire technology stack, which allegedly allows for greater efficiency than commercially available GPUs.

Meta says that while the next-gen MTIA does not replace GPUs for training or running models, it enhances them by optimising performance and power consumption.

Meta’s investment in its own AI hardware appears to be part of a larger strategy to build a more controlled and efficient ecosystem for its numerous data-intensive services and platforms.

Importantly, it also has the potential for Meta to reduce its reliance on external hardware supplies — such as the ones mentioned above — which could lead to significant cost savings.

Other AI news this week

• Google made a truckload of other AI announcements this week
• Australia’s own Leonardo.Ai formally revealed its enterprise offerings at the AWS Summit in Sydney
• The ‘smartphone replacement’ Humane AI pin is finally out… and is being panned by reviewers
• Amazon Bedrock is now available in Sydney
• The EU’s competition watchdog has major concerns around big tech’s dominance over generative AI.