The processing-using-memory (PUM) paradigm aims to eliminate data movement energy and performance costs by using memory cell interactions to directly perform computation. Given PUM’s potential for large savings, prior works have proposed many different datapath microarchitectures to demonstrate how general-purpose PUM benefits a wide range of application kernels. Ufortunately, these efforts largely depend on microarchitecturespecific vector-like interfaces that (1) force many of an application’s operations to be offloaded to a CPU, (2) require significant programmer effort to scale up applications to an entire memory chip, and (3) make it impractical to develop badly-needed systems software and programming tools for PUM.

To address these three issues, we propose the memory processing unit (MPU), a microarchitecture-agnostic interface layer for general-purpose PUM with three components. First, we develop an MPU instruction set architecture (ISA) with instructions to facilitate application scaling and task coordination. Second, we propose an ensemble execution model that coordinates execution across millions of PUM vector function units and maps to most general-purpose PUM microarchitectures. Third, we design a comprehensive MPU control path that efficiently executes MPU ISA binaries across multiple ensembles, and can enable CPUfree execution of complex end-to-end applications with PUM. We demonstrate how the MPU maps to multiple previouslyproposed PUM datapaths, and how it achieves performance/energy improvements of 1.79×/3.31× for 21 data-intensive kernels over these prior works (67×/50× vs. a modern GPU), while also achieving performance and energy improvements for the complex end-to-end applications