Efficient Incremental Training for Deep Convolutional Neural Networks

Yudong Tao, Yuexuan Tu, Mei-Ling Shyu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

While the deep convolutional neural networks (DCNNs) have shown excellent performance in various applications, such as image classification, training a DCNN model from scratch is computationally expensive and time consuming. In recent years, a lot of studies have been done to accelerate the training of DCNNs, but most of them were performed in a one-time manner. Considering the learning patterns of the human beings, people typically feel more comfortable to learn things in an incremental way and may be overwhelmed when absorbing a large amount of new information at once. Therefore, we demonstrate a new training schema that splits the whole training process into several sub-training steps. In this study, we propose an efficient DCNN training framework where we learn the new classes of concepts incrementally. The experiments are conducted on CIFAR-100 with VGG-19 as the backbone network. Our proposed framework demonstrates a comparable accuracy compared with the model trained from scratch and has shown 1.42x faster training speed.

Original languageEnglish (US)
Title of host publicationProceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages286-291
Number of pages6
ISBN (Electronic)9781728111988
DOIs
StatePublished - Apr 22 2019
Event2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 - San Jose, United States
Duration: Mar 28 2019Mar 30 2019

Publication series

NameProceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

Conference

Conference2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
CountryUnited States
CitySan Jose
Period3/28/193/30/19

Fingerprint

Neural networks
Image classification
Experiments

Keywords

  • Deep Convolutional Neural Network (DCNN)
  • Efficient Model Training
  • Incremental Model Training

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Signal Processing
  • Media Technology

Cite this

Tao, Y., Tu, Y., & Shyu, M-L. (2019). Efficient Incremental Training for Deep Convolutional Neural Networks. In Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 (pp. 286-291). [8695339] (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MIPR.2019.00058

Efficient Incremental Training for Deep Convolutional Neural Networks. / Tao, Yudong; Tu, Yuexuan; Shyu, Mei-Ling.

Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 286-291 8695339 (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tao, Y, Tu, Y & Shyu, M-L 2019, Efficient Incremental Training for Deep Convolutional Neural Networks. in Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019., 8695339, Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019, Institute of Electrical and Electronics Engineers Inc., pp. 286-291, 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019, San Jose, United States, 3/28/19. https://doi.org/10.1109/MIPR.2019.00058
Tao Y, Tu Y, Shyu M-L. Efficient Incremental Training for Deep Convolutional Neural Networks. In Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 286-291. 8695339. (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019). https://doi.org/10.1109/MIPR.2019.00058
Tao, Yudong ; Tu, Yuexuan ; Shyu, Mei-Ling. / Efficient Incremental Training for Deep Convolutional Neural Networks. Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 286-291 (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019).
@inproceedings{2d8c72d44cda45268de0ecafacead4f2,
title = "Efficient Incremental Training for Deep Convolutional Neural Networks",
abstract = "While the deep convolutional neural networks (DCNNs) have shown excellent performance in various applications, such as image classification, training a DCNN model from scratch is computationally expensive and time consuming. In recent years, a lot of studies have been done to accelerate the training of DCNNs, but most of them were performed in a one-time manner. Considering the learning patterns of the human beings, people typically feel more comfortable to learn things in an incremental way and may be overwhelmed when absorbing a large amount of new information at once. Therefore, we demonstrate a new training schema that splits the whole training process into several sub-training steps. In this study, we propose an efficient DCNN training framework where we learn the new classes of concepts incrementally. The experiments are conducted on CIFAR-100 with VGG-19 as the backbone network. Our proposed framework demonstrates a comparable accuracy compared with the model trained from scratch and has shown 1.42x faster training speed.",
keywords = "Deep Convolutional Neural Network (DCNN), Efficient Model Training, Incremental Model Training",
author = "Yudong Tao and Yuexuan Tu and Mei-Ling Shyu",
year = "2019",
month = "4",
day = "22",
doi = "10.1109/MIPR.2019.00058",
language = "English (US)",
series = "Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "286--291",
booktitle = "Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019",

}

TY - GEN

T1 - Efficient Incremental Training for Deep Convolutional Neural Networks

AU - Tao, Yudong

AU - Tu, Yuexuan

AU - Shyu, Mei-Ling

PY - 2019/4/22

Y1 - 2019/4/22

N2 - While the deep convolutional neural networks (DCNNs) have shown excellent performance in various applications, such as image classification, training a DCNN model from scratch is computationally expensive and time consuming. In recent years, a lot of studies have been done to accelerate the training of DCNNs, but most of them were performed in a one-time manner. Considering the learning patterns of the human beings, people typically feel more comfortable to learn things in an incremental way and may be overwhelmed when absorbing a large amount of new information at once. Therefore, we demonstrate a new training schema that splits the whole training process into several sub-training steps. In this study, we propose an efficient DCNN training framework where we learn the new classes of concepts incrementally. The experiments are conducted on CIFAR-100 with VGG-19 as the backbone network. Our proposed framework demonstrates a comparable accuracy compared with the model trained from scratch and has shown 1.42x faster training speed.

AB - While the deep convolutional neural networks (DCNNs) have shown excellent performance in various applications, such as image classification, training a DCNN model from scratch is computationally expensive and time consuming. In recent years, a lot of studies have been done to accelerate the training of DCNNs, but most of them were performed in a one-time manner. Considering the learning patterns of the human beings, people typically feel more comfortable to learn things in an incremental way and may be overwhelmed when absorbing a large amount of new information at once. Therefore, we demonstrate a new training schema that splits the whole training process into several sub-training steps. In this study, we propose an efficient DCNN training framework where we learn the new classes of concepts incrementally. The experiments are conducted on CIFAR-100 with VGG-19 as the backbone network. Our proposed framework demonstrates a comparable accuracy compared with the model trained from scratch and has shown 1.42x faster training speed.

KW - Deep Convolutional Neural Network (DCNN)

KW - Efficient Model Training

KW - Incremental Model Training

UR - http://www.scopus.com/inward/record.url?scp=85065604368&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065604368&partnerID=8YFLogxK

U2 - 10.1109/MIPR.2019.00058

DO - 10.1109/MIPR.2019.00058

M3 - Conference contribution

T3 - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

SP - 286

EP - 291

BT - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -