TY - JOUR
T1 - 3DCD
T2 - Scene Independent End-to-End Spatiotemporal Feature Learning Framework for Change Detection in Unseen Videos
AU - Mandal, Murari
AU - Dhar, Vansh
AU - Mishra, Abhishek
AU - Vipparthi, Santosh Kumar
AU - Abdel-Mottaleb, Mohamed
N1 - Funding Information:
Manuscript received May 21, 2020; revised August 25, 2020 and September 28, 2020; accepted October 27, 2020. Date of publication November 18, 2020; date of current version November 24, 2020. This work was supported in part by IBM of the online GPU Grant and in part by the Department of Science and Technology-Science and Engineering Research Board (DST-SERB) Project under Grant EEQ/2017/000673. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Guo-Jun Qi. (Corresponding author: Santosh Kumar Vipparthi.) Murari Mandal, Vansh Dhar, Abhishek Mishra, and Santosh Kumar Vip-parthi are with the Vision Intelligence Laboratory, Department of Computer Science and Engineering, Malaviya National Institute of Technology at Jaipur, Jaipur 302017, India (e-mail: murarimandal.cv@gmail.com; vanshd-har.ai@gmail.com; mishra.abhishek.ai@gmail.com; skvipparthi@mnit.ac.in).
Publisher Copyright:
© 1992-2012 IEEE.
PY - 2021
Y1 - 2021
N2 - Change detection is an elementary task in computer vision and video processing applications. Recently, a number of supervised methods based on convolutional neural networks have reported high performance over the benchmark dataset. However, their success depends upon the availability of certain proportions of annotated frames from test video during training. Thus, their performance on completely unseen videos or scene independent setup is undocumented in the literature. In this work, we present a scene independent evaluation (SIE) framework to test the supervised methods in completely unseen videos to obtain generalized models for change detection. In addition, a scene dependent evaluation (SDE) is also performed to document the comparative analysis with the existing approaches. We propose a fast (speed-25 fps) and lightweight (0.13 million parameters, model size-1.16 MB) end-to-end 3D-CNN based change detection network (3DCD) with multiple spatiotemporal learning blocks. The proposed 3DCD consists of a gradual reductionist block for background estimation from past temporal history. It also enables motion saliency estimation, multi-schematic feature encoding-decoding, and finally foreground segmentation through several modular blocks. The proposed 3DCD outperforms the existing state-of-the-art approaches evaluated in both SIE and SDE setup over the benchmark CDnet 2014, LASIESTA and SBMI2015 datasets. To the best of our knowledge, this is a first attempt to present results in clearly defined SDE and SIE setups in three change detection datasets.
AB - Change detection is an elementary task in computer vision and video processing applications. Recently, a number of supervised methods based on convolutional neural networks have reported high performance over the benchmark dataset. However, their success depends upon the availability of certain proportions of annotated frames from test video during training. Thus, their performance on completely unseen videos or scene independent setup is undocumented in the literature. In this work, we present a scene independent evaluation (SIE) framework to test the supervised methods in completely unseen videos to obtain generalized models for change detection. In addition, a scene dependent evaluation (SDE) is also performed to document the comparative analysis with the existing approaches. We propose a fast (speed-25 fps) and lightweight (0.13 million parameters, model size-1.16 MB) end-to-end 3D-CNN based change detection network (3DCD) with multiple spatiotemporal learning blocks. The proposed 3DCD consists of a gradual reductionist block for background estimation from past temporal history. It also enables motion saliency estimation, multi-schematic feature encoding-decoding, and finally foreground segmentation through several modular blocks. The proposed 3DCD outperforms the existing state-of-the-art approaches evaluated in both SIE and SDE setup over the benchmark CDnet 2014, LASIESTA and SBMI2015 datasets. To the best of our knowledge, this is a first attempt to present results in clearly defined SDE and SIE setups in three change detection datasets.
KW - 3D-CNN
KW - Change detection
KW - background subtraction
KW - deep learning
KW - scene independence
KW - spatiotemporal
UR - http://www.scopus.com/inward/record.url?scp=85096889863&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85096889863&partnerID=8YFLogxK
U2 - 10.1109/TIP.2020.3037472
DO - 10.1109/TIP.2020.3037472
M3 - Article
C2 - 33206604
AN - SCOPUS:85096889863
VL - 30
SP - 546
EP - 558
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
SN - 1057-7149
M1 - 9263106
ER -