GigaOm Sonar Report for Data Processing Unitsv2.0

1. Summary

A data processing unit (DPU) is a hardware accelerator, typically provided as a generic peripheral component interconnect express (PCIe) card to be installed in commodity x86 servers. A main function of DPUs is to offload specialized compute tasks from the general-purpose system central processing unit (CPU), improving the overall performance and efficiency of the entire system. These compute tasks are data-centric, frequently focusing on network and storage acceleration, compared to the graphical, floating-point and matrix-math focus of GPUs and the tensor-math focus of tensor processing units (TPUs).

DPUs help organizations build IT infrastructure that is denser, faster, more efficient, and more cost effective than alternate approaches.

Figure 1 highlights the difference between traditional servers (left) and those with DPUs (right).

Figure 1. Comparison of Traditional and DPU-Based Server Configurations

DPUs are implemented using three technologies:

• Field-programmable gate arrays (FPGAs)
• Application-specific integrated circuits (ASICs)
• Proprietary system on a chip (SoC) designs

More sophisticated implementations use a combination of these technologies, as each approach has different benefits and tradeoffs that align with different use cases. All three share some core characteristics, such as a high degree of internal parallelism to ensure consistent low latency and high throughput.

DPUs are programmable, an important feature that helps to distinguish them from other options. The relative ease with which DPUs can be modified after deployment helps them remain relevant to changing customer needs by supporting the latest protocols, algorithms, and features.

For the enterprise, it is important that the DPU integrates well with the operating system (OS), hypervisor, and other software components in the system. For more specialized workloads, such as high-performance computing (HPC) or artificial intelligence/machine learning (AI/ML), the availability of software development kits (SDKs) helps customers to customize the solution to better suit their specific needs.

DPUs accelerate a number of tasks, including network and storage functions related to data protection, encryption and security, data footprint optimization, and high availability. In some cases, DPUs provide additional functions that can replace commonly used data stores, such as key-value stores. Some vendors also include programming libraries to support compute-intensive tasks that need high degrees of parallelism and throughput.

How We Got Here

Special-purpose accelerators have long been used to augment the capabilities of general-purpose CPUs. From mainframe use of channel I/O, through math co-processors that handled floating-point operations in early PCs, to the modern GPUs, TPUs, and other specialized devices, their goal is the same: improving overall system performance.

In recent years, the growth of technologies like flash memory and high-speed networking, combined with the demand for large-scale data processing workloads, has placed additional pressure on system CPUs, making them the bottleneck. As in the past, offloading certain operations to dedicated hardware helps reduce that pressure, removing the bottleneck.

While there has been substantial progress in specialized computing accelerators, such as GPUs and TPUs, compute tasks still require access to data, and lots of it. Before DPUs became available, moving data to and from storage devices required substantial involvement from the CPU. Now, some DPUs are able to bypass the CPU entirely, moving data directly between GPUs and storage where necessary.

DPUs not only offload work from the CPU, they are also able to accelerate a variety of workloads, thanks to their programmability. While not as versatile as a general-purpose CPU, they can adapt to a wide range of common data processing tasks, unlike more limited, task-specific accelerators.