Semiconductor world – CPU/GPU Wars

Recent developments:

1. Google Cloud and NVIDIA expanded their partnership to advance AI computing, software, and services. Google Cloud already had NVIDIA GPUs, and its PaxML framework, which was originally built to span multiple Google TPU accelerator slices, will now be optimized to enable developers to use NVIDIA® H100 and A100 Tensor Core GPUs.
2. Tesla launched a new $300 million AI cluster for advanced computation. The system is equipped with 10,000 NVIDIA H100 GPUs.
3. Tesla is also building a supercomputer named Dojo, which will work in tandem with the NVIDIA H100 GPU cluster for its Full Self-Driving (FSD) ambition. Elon Musk said,

“We’ll actually take Nvidia hardware as fast as Nvidia will deliver it to us,” and “And frankly, I don’t know, if they could deliver us enough GPUs, we might not need Dojo – but they can’t.”

4. Samsung HBM3 memory and packaging technology will be utilized by AMD MI300X GPUs. NVIDIA has so far integrated HBM3 chips exclusively from SKhynix. Samsung is expected to supply ~30% of Nvidia’s HBM3 needs in 2024.
5. Intel CEO Pat Gelsinger said that the company has received “a large customer prepay” for “18A” manufacturing capacity. This is a reference to the company’s development of 1.8 nanometer production lines, which will be used to produce cutting-edge chips. Gelsinger also made a statement that Intel will be “competing more for the GPU” market.
6. Google Cloud released TPU v5e, which is suitable for midsize and large-scale AI training and inference workloads. It also announced the general availability (GA) of A3, which is powered by NVIDIA’s H100 Tensor Core GPUs.

Based on recent developments, AMD(MI300X), Google(TPU v5e), Amazon(Graviton), and Intel all are expected to produce GPUs in the next one to two years. Not sure about Meta and microsoft but Meta is building AI custom chips for metaverse ambition. These GPUs may not be as powerful as NVIDIA’s GPUs, but they should be good enough for small to medium-sized LLM training and inference workloads. By that time, NVIDIA will be able to capitalize on the demand for GPUs with huge margins and have a huge surplus that it can use to acquire other companies.