The scaling of computation throughput continues to outpace improvements in memory bandwidth, making many deep learning workloads memory-bound. Kernel fusion is a key technique to alleviate this problem, but the fusion strategies of existing compilers and frameworks are limited to using local scratchpad memory. When the intermediate results exceed the limited capacity (such as FFN), the fusion fails. Although modern GPUs (like the NVIDIA H100) now incorporate an inter-core connection mechanism known as Distributed Shared Memory (DSM)—providing a larger, high-bandwidth, and low-latency on-chip memory pool—this hardware potential has yet to be exploited by software frameworks.

To bridge this gap, we present FlashFuser, the first compiler framework to utilize inter-core connection for kernel fusion on modern GPUs. FlashFuser has three core contributions. First, we propose a powerful DSM-based communication abstraction that formalizes complex intra-cluster data exchange patterns, such as reduction, shuffle and multiply. Second, we introduce a dataflow analyzer that determines an optimal placement strategy across the entire memory hierarchy. It works by partitioning the computation into tiles and, based on a given execution order and tile sizes, calculates the data movement volume at each memory level. Finally, FlashFuser integrates these components into a unified search engine that employs an analytical model and DSM-aware heuristic rules to discover the optimal execution plan. Our evaluation on an NVIDIA H100 GPU shows that FlashFuser reduces memory access by 58% and delivers kernel speedups of 3.3x against highly-tuned libraries and 4.1x against state-of-the-art compilers, resulting in a 1.12x end-to-end speedup.