A Model for Estimating Subjective Evaluation Values of Video Degradation from Viewers’ Physiological Signals

This paper describes that it is possible to estimate a viewer’s five-level subjective evaluation of video degradation with an estimation accuracy of 0.90 or better by using physiological data such as blood volume pulses of viewers. To this end, we conducted an experiment to record participants’ EEG, BVP, gaze, pupil diameter, and subjective evaluation values of video degradation while they watched videos. We then created five different datasets from the data and built estimation models using machine learning based on random forest or neural network. As a result, the coefficient of determination for the physiological data with top importance trained by random forest was 0.997. The results contribute to an objective, continuous, unconscious, and quantitative method for estimating Quality of Experience (QoE) during video viewing.

Content

Author and article information

Contributors

Masaki Omata

Naho Kiriyama

Conference

Publication date: July 2022

Publication date (Print): July 2022

Pages: 1-10

Affiliations

[0001]University of Yamanashi

Kofu, Yamanashi, Japan

Article

DOI: 10.14236/ewic/HCI2022.23

SO-VID: 6352b5b2-5ffd-4dd7-a03b-2dddf5092009

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: 35th International BCS Human-Computer Interaction Conference

Conference acronym: HCI2022

Conference number: 35

Conference location: Keele, Staffordshire

Conference date: July 11th to 13th, 2022

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Towards a Human-Centred Digital Society

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2022.23

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

Arndt, S., Brunnström, K., Cheng, E., Engelke, U., Möller, S., and Antons, J. (2016) Review on using physiology in quality of experience. IS&T International Symposium on Electronic Imaging 2016, Human Vision and Electronic Imaging 2016, HVEI-125.
Awang, S.A., Pandiyan, P.M., Yaacob, S., Ali, Y.M., Ramidi, F., Mat, F. (2011) Spectral Density Analysis: Theta Wave as Mental Stress Indicator. Signal Processing, Image Processing and Pattern Recognition (SIP 2011). Communications in Computer and Information Science, vol 260. 103–112. Springer, Berlin, Heidelberg.
Bentaleb, A., Begen, A. C. and Zimmermann, R. (2016) SDNDASH: Improving QoE of HTTP Adaptive Streaming Using Software Defined Networking. Proceedings of the 24th ACM international conference on Multimedia (MM '16). 1296–1305. Association for Computing Machinery, New York, NY, USA.
Cao, Z., Fitschen, J. and Papadimitriou, P. (2015) FreeSurf: Application-Centric Wireless Access with SDN. Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication, 357–358. Association for Computing Machinery, New York, NY, USA.
Citi, L., Brown, E., and Barbieri, R. (2012) A Real-Time Automated Point-Process Method for the Detection and Correction of Erroneous and Ectopic Heartbeats. IEEE transactions on bio-medical engineering. vol. 59. 2828-2837.
de Winter, J. C. F. Petermeijer, S. M., Kooijman, L., and Dodou, D. (2021) Replicating five pupillometry studies of Eckhard Hess. International Journal of Psychophysiology, Vol.165, 145-205.
Gross, J., and Ligges, U. (2015) Tests for Normality. Version1.0-4, from https://cran.r-project.org/web/packages/nortest/nortest.pdf.
Hayashi, T. (2015) QoE-centric Operation for Optimizing User Quality of Experience. NTT Technical Review, Vol. 13 No. 9.
Hess, E. H., and Polt, J. M. (1960) Pupil Size as Related to Interest Value of Visual Stimuli. Science, Vol.132, No.3423, 349-350.
Hori, T. (2008) Physiological Psychology. Baifukan, Tokyo.
Huang, Z. (2015) The Research of SDN Group Policy Model Based on Application Centric Environment. 2015 Sixth International Conference on Intelligent Systems Design and Engineering Applications (ISDEA), 884-889.
International Telecommunication Union (ITU), (2019) Methodologies for the subjective assessment of the quality of television images, BT.500-14 (10/2019).
Kolb, J., Chaudhary, P., Schillinger, A., Chandra, A. and Weissman, J. (2015) Cloud-Based, User-Centric Mobile Application Optimization. IEEE International Conference on Cloud Engineering, 26-35.
Liotou, E., Tseliou, G., Samdanis, K., Tsolkas, D., Adelantado ,F., and Verikoukis, C.(2015) An SDN QoE-service for dynamically enhancing the performance of OTT applications. Seventh International Workshop on Quality of Multimedia Experience (QoMEX), 1-2.
Liotou, E., Tsolkas, D., and Passas, N. (2016) A roadmap on QoE metrics and models. 23rd International Conference on Telecommunications (ICT), 1-5.
Mitsukura, Y. (2019) KANSEI Detection and Its Application Using the EEG: Widespread of KANSEI Research in Society. IEICE ESS Fundamentals Review, Vol. 13, Issue 3, 180-186.
Miyata, Y. ed. (1998) New Physiological Psychology. 1, Kitaoji-shobo, Kyoto.
Nunez, P. L., and Srinivasan, R. (2007) Electroencephalogram. Scholarpedia, Vol. 2, No. 2:1348. Retrieved Feb. 15, 2022 from http://www.scholarpedia.org/article/Electroencephalogram.
Porcu, S., Floris, A., Voigt-Antons, J. -N., Atzori L., and Möller, S. (2020) Estimation of the Quality of Experience During Video Streaming From Facial Expression and Gaze Direction. IEEE Transactions on Network and Service Management, vol. 17, no. 4, 2702-2716.
Robitza, W. Kara, P. A. Martini, M. G., and Raake, A. (2016) On the Experimental Biases in User Behavior and QoE Assessment in the Lab. IEEE Globecom Workshops (GC Wkshps), 1-6.
Sawahata, Y., Komine, K., Hiruma, N., Itou, T., Watanabe, S., Suzuki., Y., Hara, Y., Issiki, N. (2008) Determining Relationship between Eye-Gaze Distribution and Comprehension of TV Programs. The Journal of The Institute of Image Information and Television Engineers, Vol. 62, Issue 4, 587-594.
Seddiki, M.S., Shahbaz, M., Donovan, S.P., Grover, S., Park, M., Feamster, N., and Song, Y. (2015). FlowQoS: Per-Flow Quality of Service for Broadband Access Networks. Georgia Institute of Technology, SCS Technical Report; GT-CS-15-02.
Skorin-Kapov, L., Varela, M., Hoßfeld, T., and Chen, K. (2018) A Survey of Emerging Concepts and Challenges for QoE Management of Multimedia Services. ACM Trans. Multimedia Comput. Commun. Appl. 14, 2s, Article 29.
Sony Network Communications Inc. (Retrieved Feb. 15, 2022) Neural Network Console. from https://dl.sony.com/.
Thought Technology Ltd. (Retrieved Feb. 15, 2022) EEG-Z Sensor - T9305Z. from https://thoughttechnology.com/eeg-z-sensor-t9305z/.
Thought Technology Ltd. (Retrieved Feb. 15, 2022) Blood Volume Pulse (BVP) Sensor - SA9308M. from https://thoughttechnology.com/blood-volume-pulse-bvp-sensor-sa9308m/.
Tobii AB (Retrieved Feb. 15, 2022) Tobii Pro Nano. from https://www.tobiipro.com/product-listing/nano/.
Uwano, H., Ishida K., Matsuda, Y., Fukushima, S., Nakamichi, N., Ohira, M., Matsumoto K., Okada, Y. (2008) Evaluation of Software Usability Using Electroencephalogram: Comparison of Frequency Component between Different Software Versions. Journal of Human Interface Society, human interface Vol.10 No.2 233-242.

Comments

Comment on this article

[1] Arndt, S., Brunnström, K., Cheng, E., Engelke, U., Möller, S., and Antons, J. (2016) Review on using physiology in quality of experience. IS&T International Symposium on Electronic Imaging 2016, Human Vision and Electronic Imaging 2016, HVEI-125.

[2] Awang, S.A., Pandiyan, P.M., Yaacob, S., Ali, Y.M., Ramidi, F., Mat, F. (2011) Spectral Density Analysis: Theta Wave as Mental Stress Indicator. Signal Processing, Image Processing and Pattern Recognition (SIP 2011). Communications in Computer and Information Science, vol 260. 103–112. Springer, Berlin, Heidelberg.

[3] Bentaleb, A., Begen, A. C. and Zimmermann, R. (2016) SDNDASH: Improving QoE of HTTP Adaptive Streaming Using Software Defined Networking. Proceedings of the 24th ACM international conference on Multimedia (MM '16). 1296–1305. Association for Computing Machinery, New York, NY, USA.

[4] Cao, Z., Fitschen, J. and Papadimitriou, P. (2015) FreeSurf: Application-Centric Wireless Access with SDN. Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication, 357–358. Association for Computing Machinery, New York, NY, USA.

[5] Citi, L., Brown, E., and Barbieri, R. (2012) A Real-Time Automated Point-Process Method for the Detection and Correction of Erroneous and Ectopic Heartbeats. IEEE transactions on bio-medical engineering. vol. 59. 2828-2837.

[6] de Winter, J. C. F. Petermeijer, S. M., Kooijman, L., and Dodou, D. (2021) Replicating five pupillometry studies of Eckhard Hess. International Journal of Psychophysiology, Vol.165, 145-205.

[7] Gross, J., and Ligges, U. (2015) Tests for Normality. Version1.0-4, from https://cran.r-project.org/web/packages/nortest/nortest.pdf.

[8] Hayashi, T. (2015) QoE-centric Operation for Optimizing User Quality of Experience. NTT Technical Review, Vol. 13 No. 9.

[9] Hess, E. H., and Polt, J. M. (1960) Pupil Size as Related to Interest Value of Visual Stimuli. Science, Vol.132, No.3423, 349-350.

[10] Hori, T. (2008) Physiological Psychology. Baifukan, Tokyo.

[11] Huang, Z. (2015) The Research of SDN Group Policy Model Based on Application Centric Environment. 2015 Sixth International Conference on Intelligent Systems Design and Engineering Applications (ISDEA), 884-889.

[12] International Telecommunication Union (ITU), (2019) Methodologies for the subjective assessment of the quality of television images, BT.500-14 (10/2019).

[13] Kolb, J., Chaudhary, P., Schillinger, A., Chandra, A. and Weissman, J. (2015) Cloud-Based, User-Centric Mobile Application Optimization. IEEE International Conference on Cloud Engineering, 26-35.

[14] Liotou, E., Tseliou, G., Samdanis, K., Tsolkas, D., Adelantado ,F., and Verikoukis, C.(2015) An SDN QoE-service for dynamically enhancing the performance of OTT applications. Seventh International Workshop on Quality of Multimedia Experience (QoMEX), 1-2.

[15] Liotou, E., Tsolkas, D., and Passas, N. (2016) A roadmap on QoE metrics and models. 23rd International Conference on Telecommunications (ICT), 1-5.

[16] Mitsukura, Y. (2019) KANSEI Detection and Its Application Using the EEG: Widespread of KANSEI Research in Society. IEICE ESS Fundamentals Review, Vol. 13, Issue 3, 180-186.

[17] Miyata, Y. ed. (1998) New Physiological Psychology. 1, Kitaoji-shobo, Kyoto.

[18] Nunez, P. L., and Srinivasan, R. (2007) Electroencephalogram. Scholarpedia, Vol. 2, No. 2:1348. Retrieved Feb. 15, 2022 from http://www.scholarpedia.org/article/Electroencephalogram.

[19] Porcu, S., Floris, A., Voigt-Antons, J. -N., Atzori L., and Möller, S. (2020) Estimation of the Quality of Experience During Video Streaming From Facial Expression and Gaze Direction. IEEE Transactions on Network and Service Management, vol. 17, no. 4, 2702-2716.

[20] Robitza, W. Kara, P. A. Martini, M. G., and Raake, A. (2016) On the Experimental Biases in User Behavior and QoE Assessment in the Lab. IEEE Globecom Workshops (GC Wkshps), 1-6.

[21] Sawahata, Y., Komine, K., Hiruma, N., Itou, T., Watanabe, S., Suzuki., Y., Hara, Y., Issiki, N. (2008) Determining Relationship between Eye-Gaze Distribution and Comprehension of TV Programs. The Journal of The Institute of Image Information and Television Engineers, Vol. 62, Issue 4, 587-594.

[22] Seddiki, M.S., Shahbaz, M., Donovan, S.P., Grover, S., Park, M., Feamster, N., and Song, Y. (2015). FlowQoS: Per-Flow Quality of Service for Broadband Access Networks. Georgia Institute of Technology, SCS Technical Report; GT-CS-15-02.

[23] Skorin-Kapov, L., Varela, M., Hoßfeld, T., and Chen, K. (2018) A Survey of Emerging Concepts and Challenges for QoE Management of Multimedia Services. ACM Trans. Multimedia Comput. Commun. Appl. 14, 2s, Article 29.

[24] Sony Network Communications Inc. (Retrieved Feb. 15, 2022) Neural Network Console. from https://dl.sony.com/.

[25] Thought Technology Ltd. (Retrieved Feb. 15, 2022) EEG-Z Sensor - T9305Z. from https://thoughttechnology.com/eeg-z-sensor-t9305z/.

[26] Thought Technology Ltd. (Retrieved Feb. 15, 2022) Blood Volume Pulse (BVP) Sensor - SA9308M. from https://thoughttechnology.com/blood-volume-pulse-bvp-sensor-sa9308m/.

[27] Tobii AB (Retrieved Feb. 15, 2022) Tobii Pro Nano. from https://www.tobiipro.com/product-listing/nano/.

[28] Uwano, H., Ishida K., Matsuda, Y., Fukushima, S., Nakamichi, N., Ohira, M., Matsumoto K., Okada, Y. (2008) Evaluation of Software Usability Using Electroencephalogram: Comparison of Frequency Component between Different Software Versions. Journal of Human Interface Society, human interface Vol.10 No.2 233-242.

Studying business & IT? Drive your professional career forwards with BCS books - for a 20% discount click here: shop.bcs.org