In recent earnings results, the leading cloud computing companies announced they were ramping up capex ahead of expectations as they race to get ahead in artificial intelligence (AI), but not all of this spending is going to Nvidia (US:NVDA). The semiconductor designer is the leading company in AI computing infrastructure after years of investment, but there is evidence that competitors are catching up.
The simplest way of understanding an AI supercomputer is that it’s a cluster of graphic processing units (GPUs) doing lots of big calculations. However, there are also some central processing units (CPUs) to tell the computer what to do. Plus, some DRAM (short-term) memory chips and some NAND (long-term) memory chips. The cost breakdown is roughly 50-60 per cent on GPUs, then 10-15 per cent on CPUs and DRAM chips, then 5-10 per cent on NANDs.
However, all these chips need to be connected to each other, and this can be done either with InfiniBand or Ethernet cables. This is known as ‘networking’ and is where the final 10-15 per cent of the hardware cost goes. The aim is to have connections with as much bandwidth as possible so that data can be transferred rapidly. There is no point in spending billions of dollars on GPUs if data can’t be moved quickly between them.