Timestamp aware Aberrant Detection and Analysis in Big Visual Data using Deep Learning Architecture

Science and Engineering Research Board (SERB): SERB/EEQ/2017/000673

Funding Agency: Science and Engineering Research Board, Department of Science and Technology (SERB-DST, 2018)
Principal Investigator: Dr. Santosh Kumar Vipparthi
JRF/Ph.D. Scholar: Kuldeep Marotirao Biradar
Introduction: The proposed system removes the onus of detecting aberrance situations from the manual operator; and rather, places it on the video surveillance system. The present technologies are fails to recognize aberration in video sequences. These aberrances occur over a small-time window. Thus, recognizing with its timeframe from a big visual data is really challenging task. Hence, our focus is on problems, where we are given a set of nominal training videos samples. Based on these samples need to determine whether or not a test video contains an aberration and what instant it occurs. Similarly, we aim to significantly reduce the time and human effort by automating the task and improving the accuracy by recognizing aberrances with its timestamp. Further, exploit the aberrance activity of the object by modeling the rich motion patterns in selected region, effectively capturing the underlying intrinsic structure they form in the video. Implementation of this system can be beneficial for intelligent agencies, banks, departmental stores, traffic monitoring on highway, airport terminal check-in, sports, medical field, and robotics etc.


Anomaly Detection in Traffic Videos

  Paper     PPT  

New Anamoly Dataset

A custom dataset was generated in a staged/controlled environment. We shot from four strategically placed cameras simultaneously to capture multiple views of same scene. The videos were recorded at four different locations in different times of the day. The scenes involve normal data, fight happening in different scenarios, snatching, kidnapping etc. Scenes were shot indoor/outdoor, in natural light-artificial light, low light as well to cover illumination changes. The videos were recorded from varied distances to capture subjects with varying size. Post processing yielded usable clips of approx. 90 minutes (90x60x30x4= 648000 frames). Snippet for the same are depicted in the figure.


1. Murari Mandal, Vansh Dhar, Abhishek Mishra, Santosh Kumar Vipparthi, Mohamed Abdel-Mottaleb, “3DCD: A Scene Independent End-to-End Spatiotemporal Feature Learning Framework for Change Detection in Unseen Videos,” IEEE Transactions on Image Processing, 2020, (PDF) (Impact Factor 9.34)
2. Murari Mandal, Santosh Kumar Vipparthi, “Scene Independency Matters: An Empirical Study of Scene Dependent and Scene Independent Evaluation for CNN-Based Change Detection,” IEEE Transactions on Intelligent Transportation System, 2020, (PDF) (Impact Factor 6.319)
3. Murari Mandal, Lav Kush Kumar, Santosh Kumar Vipparthi, “MOR-UAV: A Benchmark Dataset and Baselines for Moving Object Recognition in UAV Videos,” ACM Multimedia (ACMMM - 2020), (PDF) (Core - A*)
4. Murari Mandal, Vansh Dhar, Abhishek Mishra, Santosh Kumar Vipparthi, “3DFR: A Swift 3D Feature Reductionist Framework for Scene Independent Change Detection,” IEEE Signal Processing Letters, 2019 (Impact Factor 3.268).
5. Murari Mandal, Lav Kush Kumar, Mahipal Singh Saran, Santosh Kumar Vipparthi, “MotionRec: A Unified Deep Framework for Moving Object Recognition,” IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, Colorado, US, 2020
6. Monu Verma, Santosh Kumar Vipparthi, Girdhari Singh, Subrahmanyam Murala, “LEARNet: Dynamic Imaging Network for Micro Expression Recognition,” IEEE Transactions on image processing, 2019 (Impact Factor 6.79).
7. Murari Mandal, Monu Verma, Sonakshi Mathur, Santosh Vipparthi, Subrahmanyam Murala, Kranthi Deveerasetty, "RADAP: Regional Adaptive Affinitive Patterns with Logical Operators for Facial Expression Recognition," IEEE/IET Image Processing, 2019 (Impact Factor 2.004).
8.Maheep Singh, Mahesh C. Govil, Emmanuel S. Pilli, Santosh Kumar Vipparthi, "SOD-CED: salient object detection for noisy images using convolution encoder–decoder ," IEEE/IET Computer Vision, 2019 (Impact Factor 1.648)
9.Murari Mandal, Santosh Kumar Vipparthi, Mallika Chaudhary, Subramanian Murala, Anil Balaji Gonde, S. K. Nagar, "ANTIC: ANTithetic Isomeric Cluster Patterns for Medical Image Retrieval and Change Detection," IEEE/IET Computer Vision, 2018 (Impact Factor 1.648).
10. Kuldeep Marotirao Biradar, Ayushi Gupta, Murari Mandal, Santosh Kumar Vipparthi, "Challenges in Time-Stamp Aware Anomaly Detection in Traffic Videos," " In CVPR Workshops (CVPRW), Long Beach, California, US, 2019.
11.Murari Mandal, Prafulla Saxena, Santosh Kumar Vipparthi, Subrahmanyam Murala, "CANDID: Robust Change Dynamics and Deterministic Update Policy for Dynamic Background Subtraction," IEEE 24th International Conference on Pattern Recognition (ICPR), Beijing, China, 2018.
12.Monu Verma, Jaspreet Kaur Bhui, Santosh Kumar Vipparthi, Girdhari Singh, "EXPERTNet: Exigent Features Preservative Network for Facial Expression Recogntion," ACM 11th International Conference on Computer Vision, Graphics and Image Processing (ICVGIP), Hyderabad, India, 2018.
13.Kuldeep Biradar, Sachin Dube, Santosh Kumar Vipparthi, “DEAREST: Deep Convolutional Aberrant Behaviour Detection in Real world Scenario,” 13th international conference on industrial and information system, Ropar, 2018
14.Shivangi Dwivedi, Murari Mandal, Shekhar Yadav, Santosh Kumar Vipparthi, “3D CNN with Localized Residual Connections for Hyperspectral Image Classification,” 4th International Conference on Computer Vision and Image Processing (CVIP), 2019
15.Monu Verma, Prafulla Saxena, Santosh Kumar Vipparthi, Girdhari Singh, S K Nagar, “DeFINet: Portable CNN Network for Facial Expression Recognition,” IEEE International Conference on Information and Communication Technology for Competitive Strategies, 2019
16. Murari Mandal, Manal Shah, Prashant Meena, Sanhita Devi, Santosh Kumar Vipparthi, “AVDNet: A Small-Sized Vehicle Detection Network for Aerial Visual Data,” IEEE Geoscience and Remote Sensing Letters, doi: 10.1109/LGRS.2019.2923564 (Impact Factor 3.534)
17. Murari Mandal, Manal Shah, Prashant Meena, Santosh Kumar Vipparthi, “SSSDET: Simple Short and Shallow Network for Resource Efficient Vehicle Detection in Aerial Scenes.,” 26th IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 2019