Nvidia's Big Accelerator Memory: An SSD for your GPU

We're on the verge of the next generation of graphics processor units, but it's hard to stay focussed on what's right in front of us. At the end of this month, Nvidia is set to launch the hotly anticipated RTX 3090 Ti, which has a fair chance at being the most powerful gaming GPU we've ever seen. But the company has also teamed up with university researchers and IBM to unveil a new technology that already has us looking to the future of graphics cards.

According to The Register, BaM, or less excitingly, Big Accelerator Memory is Nvidia's new technology which allows a GPU to transfer to a computer's SSD without having to go through the CPU. This would allow for graphics cards to run far more independently of CPUs, as well as cut the load down on those processing chips. Conceptually it seems a bit similar to AMD's approach with the Radeon Pro SSG which had 2TB of SSD attached via M.2, which had a stronger gamer and content creation focus. 

The case paper for BaM explains the benefits of using this new system:

"We show that (1) the BaM infrastructure software running on GPUs can identify and communicate the fine-grain accesses at a sufficiently high rate to fully utilize the underlying storage devices, (2) even with consumer-grade SSDs, a BaM system can support application performance that is competitive against a much more expensive DRAM-only solution, and (3) the reduction in I/O amplification can yield significant performance benefit." 

All this means GPUs wouldn't necessarily have to rely on their own GDDR6 memory as much when it's got direct access to an SSD, and it's cutting out the CPU middle man making for faster transfers and performance. It also doesn't require a proprietary API like Microsoft's DirectStorage solution. Instead, these GPUs will transfer data using PCIe, RDMA, and a specifically designed Linux kernel driver. This lets SSDs, even consumer grade ones, read and write directly to the GPU. All in all, this sounds like a win for computing performance across the board.

Being a GPU based development, it's easy to have a gaming centric view of this technology but that's not necessarily the case. While it's very likely that the improvements in efficiency will lead to boons for PC gaming, where this is really likely to shine is for large complex workloads. These tasks are where the GPU having direct access to tonnes of SSD storage is most likely to benefit. But that doesn't mean I don't also want to see it render some very pretty videogames.