Challenges in energy-efficient deep neural network training with FPGA

Yudong Tao, Rui Ma, Mei Ling Shyu, Shu Ching Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In recent years, it is highly demanding to deploy Deep Neural Networks (DNNs) on edge devices, such as mobile phones, drones, robotics, and wearable devices, to process visual data collected by the cameras embedded in these systems. In addition to the model inference, training DNNs locally can benefit model customization and data privacy protection. Since many edge systems are powered by batteries or have limited energy budgets, Field-Programmable Gate Array (FPGA) is commonly used as the primary processing engine to satisfy both demands in performance and energy-efficiency. Although many recent research papers have been published on the topic of DNN inference with FPGAs, training a DNN with FPGAs has not been well exploited by the community. This paper summarizes the current status of adopting FPGA for DNN computation and identifies the main challenges in deploying DNN training on FPGAs. Moreover, a performance metric and evaluation workflow are proposed to compare the FPGA-based systems for DNN training in terms of (1) usage of on-chip resources, (2) training efficiency, (3) energy efficiency, and (4) model performance for specific computer vision tasks.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2020
PublisherIEEE Computer Society
Pages1602-1611
Number of pages10
ISBN (Electronic)9781728193601
DOIs
StatePublished - Jun 2020
Event2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2020 - Virtual, Online, United States
Duration: Jun 14 2020Jun 19 2020

Publication series

NameIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
Volume2020-June
ISSN (Print)2160-7508
ISSN (Electronic)2160-7516

Conference

Conference2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2020
CountryUnited States
CityVirtual, Online
Period6/14/206/19/20

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Challenges in energy-efficient deep neural network training with FPGA'. Together they form a unique fingerprint.

  • Cite this

    Tao, Y., Ma, R., Shyu, M. L., & Chen, S. C. (2020). Challenges in energy-efficient deep neural network training with FPGA. In Proceedings - 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2020 (pp. 1602-1611). [9151041] (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops; Vol. 2020-June). IEEE Computer Society. https://doi.org/10.1109/CVPRW50498.2020.00208