Deep Learning based Human Detection in Privacy-Preserved Surveillance Videos

Visual surveillance systems have been improving rapidly over the recent past, becoming more capable and pervasive with incorporation of artificial intelligence. At the same time such surveillance systems are exposing the public to new privacy and security threats. There have been an increasing number of reports of blatant abuse of surveillance technologies. To counteract this, data privacy regulations (e.g. GDPR in Europe) have provided guidelines for data collection and data processing. However, there is still a need for a private and secure method of model training for advanced machine learning and deep learning algorithms. To this end, in this paper we propose a privacy-preserved method for visual surveillance. We first develop a dataset of privacy preserved videos. The data in these videos is masked using Gaussian Mixture Model (GMM) and selective encryption. We then train high-performance object detection models on the generated dataset. The proposed method utilizes state-of-art object detection deep learning models (viz. YOLO v4 and YOLO v5) to perform human/object detection in masked videos. The results are encouraging, and are pointers to the viability of the use of modern day deep learning models for object detection in privacy-preserved videos.

Content

Author and article information

Contributors

Muhammad Jehanzaib Yousuf

Nadia Kanwal

Mohammad Samar Ansari

Mamoona Asghar

Brian Lee

Conference

Publication date: July 2022

Publication date (Print): July 2022

Pages: 1-7

Affiliations

[0001]Software Research Institute

TU Shannon: Midlands Midwest

Athlone, Ireland

[0002]School of Computing and Mathematics

Keele University

United Kingdom

[0003]Faculty of Science and Engineering

University of Chester

United Kingdom

[0004]School of Computer Science

National University of Ireland Galway

Ireland

Article

DOI: 10.14236/ewic/HCI2022.33

SO-VID: 70a446be-5f2d-4e8b-8c0b-b66673a9ac17

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: 35th International BCS Human-Computer Interaction Conference

Conference acronym: HCI2022

Conference number: 35

Conference location: Keele, Staffordshire

Conference date: July 11th to 13th, 2022

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Towards a Human-Centred Digital Society

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2022.33

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

Abbasi, M. H., Majidi, B., Eshghi, M. and Abbasi, E. H. (2019), Deep visual privacy preserving for internet of robotic things, in ‘2019 5th Conference on Knowledge Based Engineering and Innovation
(KBEI)’, pp. 292–296.
Asghar, M. N., Kanwal, N., Lee, B., Fleury, M., Herbst, M. and Qiao, Y. (2019), ‘Visual surveillance within the eu general data protection regulation: A technology perspective’, IEEE Access 7, 111709–111726.
Aslam, A. and Curry, E. (2021), ‘A survey on object detection for the internet of multimedia things (iomt) using deep learning and eventbased middleware: approaches, challenges, and future directions’, Image and Vision Computing 106, 104095.
BBC News (06 Dec,2005), ‘CCTV staff ’spied on naked woman”, BBC News. URL: http://news.bbc.co.uk/2/hi/uknews/england/merseyside/4503244.stm
Bochkovskiy, A., Wang, C.-Y. and Liao, H.-Y. M. (2020), ‘Yolov4: Optimal speed and accuracy of object detection’, ArXiv abs/2004.10934.
Buyya, R., Selvi, S. T. and Chu, X. (2009), Object-oriented programming with Java: essentials and applications, Tata McGraw-Hill.
Cosgrove, E. (2019), ‘One billion surveillance cameras will be watching around the world in 2021, a new study says’, CNBC. URL: https://www.cnbc.com/2019/12/06/onebillion-surveillance-cameras-will-be-watchingglobally-in-2021.html
Cournan, M., Fusco-Gessick, B. and Wright, L. (2018), ‘Improving patient safety through video monitoring’, Rehabilitation Nursing 43, 111–115.
Fitwi, A. and Chen, Y. (2020), Privacy-preserving selective video surveillance, in ‘2020 29th International Conference on Computer Communications and Networks (ICCCN)’, pp. 1–10.
Fitwi, A., Chen, Y. and Zhu, S. (2020), Prise: Slenderized privacy-preserving surveillance as an edge service, in ‘2020 IEEE 6th International Conference on Collaboration and Internet Computing (CIC)’, pp. 125–134.
Furlong, R. (2006), ‘Germans probe merkel spy camera’, BBC News. URL: http://news.bbc.co.uk/2/hi/europe/4849806.stm
Geiger, D. (2021), ‘ADT technician hacked security cameras at hundreds of homes, spied on ‘attractive’ women and couples’, Oxygen-True Crime. URL: https://www.oxygen.com/crimenews/telesforo-aviles-admits-to-hacking-adthome-security-cameras
Harvey, Adam. LaPlace, J. (2021), ‘Exposing.ai’. URL: https://exposing.ai/dukemtmc/
Humphrys, J. (16 August, 2019), ‘John humphrys - facial recognition cameras: How worried should we be?’, youGov. URL: https://yougov.co.uk/topics/politics/articlesreports/2019/08/16/john-humphrys-facialrecognition-cameras-how-worri
Ivasic-Kos, M., Iosifidis, A., Tefas, A. and Pitas, I. (2014), Person de-identification in activity videos, in ‘2014 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO)’, pp. 1294–1299.
Jocher, G. (2020), ‘Yolov5’, GitHub. URL: https://github.com/ultralytics/yolov5
Kim, M., Jiang, X., Lauter, K., Ismayilzada, E. and Shams, S. (2021), ‘Hear: Human action recognition via neural networks on homomorphically encrypted data’.
Liu, J., Tan, R., Han, G., Sun, N. and Kwong, S. (2021), ‘Privacy-preserving in-home fall detection using visual shielding sensing and private information-embedding’, IEEE Transactions on Multimedia 23, 3684–3699.
Meye, C. (2021), ‘Breach of 150,000 surveillance cameras sparks credential concerns’, BBC News. URL: https://www.asisonline.org/securitymanagement-magazine/latest-news/todayin-security/2021/march/breach-surveillancecameras-sparks-credential-concerns
Pillai, G. (2012), ‘Caught on camera: You are filmed on CCTV 300 times a day in london’, International Business Times. URL: https://www.ibtimes.co.uk/britain-cctvcamera-surveillance-watch-london-big-312382
Redmon, J., Divvala, S. K., Girshick, R. B. and Farhadi, A. (2015), ‘You only look once: Unified, real-time object detection’, CoRR abs/1506.02640.
Richards, N. M. (2013), ‘The Dangers of Surveillance’, Harvard Law Review 127, 1934–1965.
Ryoo, M. S., Rothrock, B., Fleming, C. and Yang, H. J. (2017), Privacy-preserving human activity recognition from extreme low resolution, in ‘Thirty-First AAAI Conference on Artificial Intelligence’.
Semertzidis, T., K.Dimitropoulos, A.Koutsia and N.Grammalidis (2010), ‘Video sensor network for real-time traffic monitoring and surveillance’, IET Intelligent Transport Systems 4, 103–112(9).
Tariq, F., Kanwal, N., Ansari, M. S., Afzaal, A., Asghar, M. N. and Anjum, M. J. (2020), Towards a privacy preserving surveillance approach for smart cities, in ‘3rd Smart Cities Symposium (SCS 2020)’, Vol. 2020, pp. 450–455.
Zaidi, S. S. A., Ansari, M. S., Aslam, A., Kanwal, N., Asghar, M. and Lee, B. (2022), ‘A survey of modern deep learning based object detection models’, Digital Signal Processing 126, 103514.

Comments

Comment on this article

[1] Abbasi, M. H., Majidi, B., Eshghi, M. and Abbasi, E. H. (2019), Deep visual privacy preserving for internet of robotic things, in ‘2019 5th Conference on Knowledge Based Engineering and Innovation

[2] (KBEI)’, pp. 292–296.

[3] Asghar, M. N., Kanwal, N., Lee, B., Fleury, M., Herbst, M. and Qiao, Y. (2019), ‘Visual surveillance within the eu general data protection regulation: A technology perspective’, IEEE Access 7, 111709–111726.

[4] Aslam, A. and Curry, E. (2021), ‘A survey on object detection for the internet of multimedia things (iomt) using deep learning and eventbased middleware: approaches, challenges, and future directions’, Image and Vision Computing 106, 104095.

[5] BBC News (06 Dec,2005), ‘CCTV staff ’spied on naked woman”, BBC News. URL: http://news.bbc.co.uk/2/hi/uknews/england/merseyside/4503244.stm

[6] Bochkovskiy, A., Wang, C.-Y. and Liao, H.-Y. M. (2020), ‘Yolov4: Optimal speed and accuracy of object detection’, ArXiv abs/2004.10934.

[7] Buyya, R., Selvi, S. T. and Chu, X. (2009), Object-oriented programming with Java: essentials and applications, Tata McGraw-Hill.

[8] Cosgrove, E. (2019), ‘One billion surveillance cameras will be watching around the world in 2021, a new study says’, CNBC. URL: https://www.cnbc.com/2019/12/06/onebillion-surveillance-cameras-will-be-watchingglobally-in-2021.html

[9] Cournan, M., Fusco-Gessick, B. and Wright, L. (2018), ‘Improving patient safety through video monitoring’, Rehabilitation Nursing 43, 111–115.

[10] Fitwi, A. and Chen, Y. (2020), Privacy-preserving selective video surveillance, in ‘2020 29th International Conference on Computer Communications and Networks (ICCCN)’, pp. 1–10.

[11] Fitwi, A., Chen, Y. and Zhu, S. (2020), Prise: Slenderized privacy-preserving surveillance as an edge service, in ‘2020 IEEE 6th International Conference on Collaboration and Internet Computing (CIC)’, pp. 125–134.

[12] Furlong, R. (2006), ‘Germans probe merkel spy camera’, BBC News. URL: http://news.bbc.co.uk/2/hi/europe/4849806.stm

[13] Geiger, D. (2021), ‘ADT technician hacked security cameras at hundreds of homes, spied on ‘attractive’ women and couples’, Oxygen-True Crime. URL: https://www.oxygen.com/crimenews/telesforo-aviles-admits-to-hacking-adthome-security-cameras

[14] Harvey, Adam. LaPlace, J. (2021), ‘Exposing.ai’. URL: https://exposing.ai/dukemtmc/

[15] Humphrys, J. (16 August, 2019), ‘John humphrys - facial recognition cameras: How worried should we be?’, youGov. URL: https://yougov.co.uk/topics/politics/articlesreports/2019/08/16/john-humphrys-facialrecognition-cameras-how-worri

[16] Ivasic-Kos, M., Iosifidis, A., Tefas, A. and Pitas, I. (2014), Person de-identification in activity videos, in ‘2014 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO)’, pp. 1294–1299.

[17] Jocher, G. (2020), ‘Yolov5’, GitHub. URL: https://github.com/ultralytics/yolov5

[18] Kim, M., Jiang, X., Lauter, K., Ismayilzada, E. and Shams, S. (2021), ‘Hear: Human action recognition via neural networks on homomorphically encrypted data’.

[19] Liu, J., Tan, R., Han, G., Sun, N. and Kwong, S. (2021), ‘Privacy-preserving in-home fall detection using visual shielding sensing and private information-embedding’, IEEE Transactions on Multimedia 23, 3684–3699.

[20] Meye, C. (2021), ‘Breach of 150,000 surveillance cameras sparks credential concerns’, BBC News. URL: https://www.asisonline.org/securitymanagement-magazine/latest-news/todayin-security/2021/march/breach-surveillancecameras-sparks-credential-concerns

[21] Pillai, G. (2012), ‘Caught on camera: You are filmed on CCTV 300 times a day in london’, International Business Times. URL: https://www.ibtimes.co.uk/britain-cctvcamera-surveillance-watch-london-big-312382

[22] Redmon, J., Divvala, S. K., Girshick, R. B. and Farhadi, A. (2015), ‘You only look once: Unified, real-time object detection’, CoRR abs/1506.02640.

[23] Richards, N. M. (2013), ‘The Dangers of Surveillance’, Harvard Law Review 127, 1934–1965.

[24] Ryoo, M. S., Rothrock, B., Fleming, C. and Yang, H. J. (2017), Privacy-preserving human activity recognition from extreme low resolution, in ‘Thirty-First AAAI Conference on Artificial Intelligence’.

[25] Semertzidis, T., K.Dimitropoulos, A.Koutsia and N.Grammalidis (2010), ‘Video sensor network for real-time traffic monitoring and surveillance’, IET Intelligent Transport Systems 4, 103–112(9).

[26] Tariq, F., Kanwal, N., Ansari, M. S., Afzaal, A., Asghar, M. N. and Anjum, M. J. (2020), Towards a privacy preserving surveillance approach for smart cities, in ‘3rd Smart Cities Symposium (SCS 2020)’, Vol. 2020, pp. 450–455.

[27] Zaidi, S. S. A., Ansari, M. S., Aslam, A., Kanwal, N., Asghar, M. and Lee, B. (2022), ‘A survey of modern deep learning based object detection models’, Digital Signal Processing 126, 103514.

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65