cfaed Publications
Pruning and Early-Exit Co-Optimization for CNN Acceleration on FPGAs
Reference
Guilherme Korol, Michael Guilherme Jordan, Mateus Beck Rutzig, Jeronimo Castrillon, Antonio Carlos Schneider Beck, "Pruning and Early-Exit Co-Optimization for CNN Acceleration on FPGAs", Proceedings of the 2023 Design, Automation and Test in Europe Conference (DATE), IEEE, pp. 1-6, Apr 2023. [doi]
Abstract
The challenge of processing heavy-load ML tasks, particularly CNN-based ones at resource-constrained IoT devices, has encouraged the use of edge servers. The edge offers performance levels higher than the end devices and better latency and security levels than the Cloud. On top of that, the rising complexity of ML applications, the ever-increasing number of connected devices, and the current demands for energy efficiency require optimizing such CNN models. Pruning and early-exit are notable optimizations that have been successfully used to alleviate the computational cost of inference. However, these optimizations have not yet been exploited simultaneously: while pruning is usually applied at design time, which involves retraining the CNN before deployment, early-exit is inherently dynamic. In this work, we propose AdaPEx, a framework that exploits the intrinsic reconfigurable FPGA capabilities so both can be cooperatively employed. AdaPEx first explores the trade-off between pruning and early-exit at design-time, creating a design space never exploited in the state-of-the-art. Then, AdaPEx applies FPGA reconfiguration as a means to enable the combined use of pruning and early-exit dynamically. At runtime, this allows matching the inference processing to the current edge conditions and a user-configurable accuracy threshold. In a smart IoT application, AdaPEx processes up to 1.32x more inferences and improves EDP by up to 2.55x over the state-of-the-art FPGA-based FINN accelerator.
Bibtex
author = {Guilherme Korol and Michael Guilherme Jordan and Mateus Beck Rutzig and Jeronimo Castrillon and Antonio Carlos Schneider Beck},
booktitle = {Proceedings of the 2023 Design, Automation and Test in Europe Conference (DATE)},
title = {Pruning and Early-Exit Co-Optimization for CNN Acceleration on FPGAs},
location = {Antwerp, Belgium},
pages = {6pp},
publisher = {IEEE},
series = {DATE'23},
month = apr,
year = {2023},
doi = {10.23919/DATE56975.2023.10137244},
pages = {1-6},
url = {https://ieeexplore.ieee.org/document/10137244},
abstract = {The challenge of processing heavy-load ML tasks, particularly CNN-based ones at resource-constrained IoT devices, has encouraged the use of edge servers. The edge offers performance levels higher than the end devices and better latency and security levels than the Cloud. On top of that, the rising complexity of ML applications, the ever-increasing number of connected devices, and the current demands for energy efficiency require optimizing such CNN models. Pruning and early-exit are notable optimizations that have been successfully used to alleviate the computational cost of inference. However, these optimizations have not yet been exploited simultaneously: while pruning is usually applied at design time, which involves retraining the CNN before deployment, early-exit is inherently dynamic. In this work, we propose AdaPEx, a framework that exploits the intrinsic reconfigurable FPGA capabilities so both can be cooperatively employed. AdaPEx first explores the trade-off between pruning and early-exit at design-time, creating a design space never exploited in the state-of-the-art. Then, AdaPEx applies FPGA reconfiguration as a means to enable the combined use of pruning and early-exit dynamically. At runtime, this allows matching the inference processing to the current edge conditions and a user-configurable accuracy threshold. In a smart IoT application, AdaPEx processes up to 1.32x more inferences and improves EDP by up to 2.55x over the state-of-the-art FPGA-based FINN accelerator.},
}
Downloads
2304_Korol_DATE [PDF]
Permalink
https://cfaed.tu-dresden.de/publications?pubId=3438