Cocco: Hardware-Mapping Co-Exploration towards Memory Capacity-Communication Optimization
Abstract
Memory is a critical design consideration in current data-intensive DNN accelerators, profoundly affecting energy consumption, bandwidth requirements, and area costs. With the growing complexity of structural design, the on-chip memory capacity should get larger to ease the system burden of data movement at the expense of silicon costs. Although several previous works have discussed foundational memory-oriented optimizations like various data reuse manners, they are difficult to provide a universal and potent methodology for coping with diverse graph structures. This paper presents a holistic discussion on the intrinsic connection between a network structure and memory features to guide the optimization from both hardware and mapping perspectives. First, we present a graph-level execution scheme with the corresponding dataflow and memory management method. This scheme enables the execution of an arbitrary graph pattern with high data reuse and low hardware overhead. Subsequently, we propose Cocco, a hardware-mapping co-exploration framework based on graph-level features of networks. It aims to achieve lower communication overhead (i.e., energy consumption and bandwidth requirements) under a smaller memory capacity. We formulate the graph-partition scheduling and memory configuration search as an optimization problem and employ a genetic-based method to achieve efficient co-exploration for large and irregular networks. Experiments demonstrate that Cocco provides lower external memory access, lower bandwidth requirements, and more stable optimization for graph partition compared to the greedy algorithm and dynamic programming introduced in prior works. Cocco also presents 1.89% to 50.33% lower costs using co-exploration in contrast to other methods
