Abstract
While the deep convolutional neural networks (DCNNs) have shown excellent performance in various applications, such as image classification, training a DCNN model from scratch is computationally expensive and time consuming. In recent years, a lot of studies have been done to accelerate the training of DCNNs, but most of them were performed in a one-time manner. Considering the learning patterns of the human beings, people typically feel more comfortable to learn things in an incremental way and may be overwhelmed when absorbing a large amount of new information at once. Therefore, we demonstrate a new training schema that splits the whole training process into several sub-training steps. In this study, we propose an efficient DCNN training framework where we learn the new classes of concepts incrementally. The experiments are conducted on CIFAR-100 with VGG-19 as the backbone network. Our proposed framework demonstrates a comparable accuracy compared with the model trained from scratch and has shown 1.42x faster training speed.
| Original language | English (US) |
|---|---|
| Title of host publication | Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 286-291 |
| Number of pages | 6 |
| ISBN (Electronic) | 9781728111988 |
| DOIs | |
| State | Published - Apr 22 2019 |
| Event | 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 - San Jose, United States Duration: Mar 28 2019 → Mar 30 2019 |
Publication series
| Name | Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 |
|---|
Conference
| Conference | 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 |
|---|---|
| Country | United States |
| City | San Jose |
| Period | 3/28/19 → 3/30/19 |
Fingerprint
Keywords
- Deep Convolutional Neural Network (DCNN)
- Efficient Model Training
- Incremental Model Training
ASJC Scopus subject areas
- Computer Networks and Communications
- Signal Processing
- Media Technology
Cite this
Efficient Incremental Training for Deep Convolutional Neural Networks. / Tao, Yudong; Tu, Yuexuan; Shyu, Mei-Ling.
Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 286-291 8695339 (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Efficient Incremental Training for Deep Convolutional Neural Networks
AU - Tao, Yudong
AU - Tu, Yuexuan
AU - Shyu, Mei-Ling
PY - 2019/4/22
Y1 - 2019/4/22
N2 - While the deep convolutional neural networks (DCNNs) have shown excellent performance in various applications, such as image classification, training a DCNN model from scratch is computationally expensive and time consuming. In recent years, a lot of studies have been done to accelerate the training of DCNNs, but most of them were performed in a one-time manner. Considering the learning patterns of the human beings, people typically feel more comfortable to learn things in an incremental way and may be overwhelmed when absorbing a large amount of new information at once. Therefore, we demonstrate a new training schema that splits the whole training process into several sub-training steps. In this study, we propose an efficient DCNN training framework where we learn the new classes of concepts incrementally. The experiments are conducted on CIFAR-100 with VGG-19 as the backbone network. Our proposed framework demonstrates a comparable accuracy compared with the model trained from scratch and has shown 1.42x faster training speed.
AB - While the deep convolutional neural networks (DCNNs) have shown excellent performance in various applications, such as image classification, training a DCNN model from scratch is computationally expensive and time consuming. In recent years, a lot of studies have been done to accelerate the training of DCNNs, but most of them were performed in a one-time manner. Considering the learning patterns of the human beings, people typically feel more comfortable to learn things in an incremental way and may be overwhelmed when absorbing a large amount of new information at once. Therefore, we demonstrate a new training schema that splits the whole training process into several sub-training steps. In this study, we propose an efficient DCNN training framework where we learn the new classes of concepts incrementally. The experiments are conducted on CIFAR-100 with VGG-19 as the backbone network. Our proposed framework demonstrates a comparable accuracy compared with the model trained from scratch and has shown 1.42x faster training speed.
KW - Deep Convolutional Neural Network (DCNN)
KW - Efficient Model Training
KW - Incremental Model Training
UR - http://www.scopus.com/inward/record.url?scp=85065604368&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85065604368&partnerID=8YFLogxK
U2 - 10.1109/MIPR.2019.00058
DO - 10.1109/MIPR.2019.00058
M3 - Conference contribution
AN - SCOPUS:85065604368
T3 - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
SP - 286
EP - 291
BT - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
PB - Institute of Electrical and Electronics Engineers Inc.
ER -