AI-based artistic representation of emotions from EEG signals: a discussion on fairness, inclusion, and aesthetics

While Artificial Intelligence (AI) technologies are being progressively developed, artists and researchers are investigating their role in artistic practices. In this work, we present an AI-based Brain-Computer Interface (BCI) in which humans and machines interact to express feelings artistically. This system and its production of images give opportunities to reflect on the complexities and range of human emotions and their expressions. In this discussion, we seek to understand the dynamics of this interaction to reach better co-existence in fairness, inclusion, and aesthetics.

Content

Author and article information

Contributors

Piera Riccio

Kristin Bergaust

Boel Christensen-Scheel

Juan-Carlos De Martin

Maria A. Zuluaga

Stefano Nichele

Conference

Publication date: September 2021

Publication date (Print): September 2021

Pages: 336-343

Affiliations

[0001]ELLIS Unit Alicante Foundation

Oslo Metropolitan University

Alicante, Spain

[0002]Oslo Metropolitan University

Oslo, Norway

[0003]Oslo National Academy of the Arts

Oslo, Norway

[0004]Politecnico di Torino

Torino, Italy

[0005]EURECOM

Biot, France

Article

DOI: 10.14236/ewic/POM2021.45

SO-VID: 57cb6878-0e72-4779-b48b-159e5243ce4a

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of Politics of the Machines - Rogue Research 2021

Conference acronym: POM 2021

Conference number: 3

Conference location: Berlin, Germany

Conference date: September 14-17, 2021

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: debate and devise concepts and practices that seek to critically question and unravel novel modes of science

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/POM2021.45

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

Advanced Robotics and its Social Impacts (ARSO) IEEE, 1–7.
K Bergaust & S Nichele FeLT-The Futures of Living Technologies. Proceedings of POM, Beirut, 11 – 14 June 2019 , 90–97. BCS Learning and Development, Swindon.
P Booth, et al. Artountability: Art and Algorithmic Accountability. Data Protection and Privacy, Volume 14: Enforcing Rights in a Changing World, 14. Proceedings of the 14. International Conference Computers, Privacy and Data Protection – CPDP 2021, 27 - 29 January 2021 , Brussels, 45–66. Hart Publishing, Oxford.
S Colton, M. F Valstar, & M Pantic (2008, September). Emotionally aware automated portrait painting. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts, 304–311.
R. M Drag, & M. E Shaw (1967) Factors Influencing the Communication of Emotional Intent by Facial Expressions. Psychonomic Science, 8(4), 137–138.
P Ekman (1992) An Argument for Basic Emotions. Cognition & Emotion, 6(3-4), 169–200.
G Ekster (2018). Cognichrome. Available from: http://www.cognichrome.com/ ( 27 August 2021 )
I Goodfellow, Pouget-J Abadie, M Mirza, B Xu, Warde-D Farley, S Ozair, Courville., A., & Y Bengio (2014) Generative adversarial nets. Advances in neural information processing systems. NIPS’14: Proceedings of the 27. International Conference on Neural Information Processing Systems (2). Montreal, 8 – 13 December 2014 , 2672–2680. The MIT Press, Cambridge MA.
Hall, Melinda C. (2019) Critical Disability Theory. The Stanford Encyclopedia of Philosophy (Winter 2019 Edition). Available from http://plato.stanford.edu/archives/win2019/entries/disability-critical/ ( 27 August 2021 )
S Hareli, K Kafetsios, & U Hess (2015) A Cross-cultural Study on Emotion Expression and the Learning of Social Norms. Frontiers in psychology, 6, 1501.
A Howard, C Zhang, & E Horvitz (2017, March). Addressing bias in machine learning algorithms: A pilot study on emotion recognition for intelligent systems. 2017 IEEE Workshop on Systems. University of Oxford Connected Life 2018 – Conference Proceedings, 2334.
M. H Immordino-Yang, X. F Yang, & H Damasio (2016) Cultural Modes of Expressing Emotions Influence How Emotions are Experienced. Emotion, 16(7), 1033.
T Karras, M Aittala, J Hellsten, S Laine, J Lehtinen, & T Aila (2020). Training Generative Adversarial Networks with Limited Data. In IEEE Conference on Neural Information Processing Systems.
N Kordzadeh, & M Ghasemaghaei (2021) Algorithmic Bias: Review, Synthesis, and Future Research Directions. European Journal of Information Systems, 1–22.
P. J Lang, M. M Bradley, & B. N Cuthbert (1997) International Affective Picture System (IAPS): Technical Manual and Affective Ratings. NIMH Center for the Study of Emotion and Attention, 1(39-58), 3.
M Levin, & T Siebers (2010) The Art of Disability: An Interview with Tobin Siebers. Disability Studies Quarterly, 30(2).
A Maffei, & A Angrilli (2019) E-MOVIE-Experimental Movies for Induction Of Emotions In Neuroscience: An Innovative Film Database With Normative Data And Sex Differences. Plos One, 14(10).
L Manovich (2017) Automating aesthetics: Artificial intelligence and image culture. Flash Art International, 316, 1–10.
J McCormack, & d’M Inverno (2014). On the future of computers and creativity.Proceedings of AISB 2014: Symposium on Computational Creativity, London, 1 – 4 April 2014 .
J McCormack, O Bown, A Dorin, J McCabe, G Monro, & M Whitelaw (2014) Ten Questions Concerning Generative Computer Art. Leonardo, 47(2), 135–141.
M McLuhan, & Q Fiore (1967) The Medium Is the Message. New York, 123, 126–128.
S Mohammad, & S Kiritchenko (2018, May). Wikiart emotions: An annotated dataset of emotions evoked by art. Proceedings of the eleventh international conference on language resources and evaluation LREC 2018.
S Ogolla & A Gupta (2018). Inclusive Design – Methods to Ensure A High Degree Of Participation. In: Artificial Intelligence (AI)
R. W Picard (1995). Affective Computing. In M.I.T Media Laboratory Perceptual Computing Section Technical Report No. 321.
R. W Picard (2000). Affective Computing for HCI. HCI, 1, 829–833.
Random Quark (2017). The Art of Feeling. Available from : http://randomquark.com/case-studies/mindswarms.html/ ( 27 August 2021 )
P Riccio Design of a Brain-Computer Interface to Translate Emotional States Into Original Paintings. Diss. Politecnico di Torino, 2021.
J. A Russell (1980). A Circumplex Model of Affect. Journal Of Personality and Social Psychology, 39(6), 1161.
S Salevati, & S DiPaola (2015). A creative artificial intelligence system to investigate user experience, affect, emotion and creativity. Electronic Visualisation and the Arts (EVA 2015), 140–147.
J Salminen, S. G Jung, & B. J Jansen (2019, May). Detecting demographic bias in automatically generated personas. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 1–6.
A Schaefer, F Nils, X Sanchez, & P Philippot (2010). Assessing The Effectiveness of a Large Database of Emotion-Eliciting Films: A New Tool for Emotion Researchers. Cognition And Emotion, 24(7), 1153–1172.
K Sherwood (2019). Out of the Blue: Art, Disability, and Yelling. In Contemporary Art and Disability Studies (pp. 213–224). Routledge.
T Siebers (2005). Disability Aesthetics. PMLA/Publications of the Modern Language Association of America, 120(2), 542–546.
P. K Solvang (2018). Between Art Therapy and Disability Aesthetics: A Sociological Approach for Understanding the Intersection Between Art Practice and Disability Discourse. Disability & Society, 33(2), 238–253.
L Suchman, & J Weber (2016). Human-Machine Autonomies. Autonomous weapons systems: Law, Ethics, Policy, 75–102.
W Sun, & Z Chen (2020). Learned Image Downscaling for Upscaling Using Content Adaptive Resampler. IEEE Transactions on Image Processing, 29, 4027–4040.
N Watson, & S Vehmas (2019). Disability Studies: Into the multidisciplinary future. N Watson, S Vehmas (Eds.) Routledge Handbook of Disability Studies. Routledge, London.
A Wierzbicka (1999). Emotions across languages and cultures: Diversity and universals. Cambridge University Press, Cambridge
W Yang, K Makita, T Nakao, N Kanayama, M. G Machizawa, T Sasaoka, A Sugata, R Kobayashi, R Hiramoto, S Yamawaki, M Iwanaga, & M Miyatani (2018). Affective Auditory Stimulus Database: An Expanded Version of the International Affective Digitized Sounds (IADS-E). Behavior Research Methods, 50(4), 1415–1429.
W. L Zheng, W Liu, Y Lu, B. L Lu, & A Cichocki (2018). Emotionmeter: A Multimodal Framework for Recognizing Human Emotions. IEEE Transactions on Cybernetics, 49(3), 1110–1122.
P Zhong, D Wang, & C Miao (2020). EEG-based Emotion Recognition Using Regularized Graph Neural Networks. IEEE Transactions on Affective Computing, 1.

Comments

Comment on this article

[1] Advanced Robotics and its Social Impacts (ARSO) IEEE, 1–7.

[2] K Bergaust & S Nichele FeLT-The Futures of Living Technologies. Proceedings of POM, Beirut, 11 – 14 June 2019 , 90–97. BCS Learning and Development, Swindon.

[3] P Booth, et al. Artountability: Art and Algorithmic Accountability. Data Protection and Privacy, Volume 14: Enforcing Rights in a Changing World, 14. Proceedings of the 14. International Conference Computers, Privacy and Data Protection – CPDP 2021, 27 - 29 January 2021 , Brussels, 45–66. Hart Publishing, Oxford.

[4] S Colton, M. F Valstar, & M Pantic (2008, September). Emotionally aware automated portrait painting. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts, 304–311.

[5] R. M Drag, & M. E Shaw (1967) Factors Influencing the Communication of Emotional Intent by Facial Expressions. Psychonomic Science, 8(4), 137–138.

[6] P Ekman (1992) An Argument for Basic Emotions. Cognition & Emotion, 6(3-4), 169–200.

[7] G Ekster (2018). Cognichrome. Available from: http://www.cognichrome.com/ ( 27 August 2021 )

[8] I Goodfellow, Pouget-J Abadie, M Mirza, B Xu, Warde-D Farley, S Ozair, Courville., A., & Y Bengio (2014) Generative adversarial nets. Advances in neural information processing systems. NIPS’14: Proceedings of the 27. International Conference on Neural Information Processing Systems (2). Montreal, 8 – 13 December 2014 , 2672–2680. The MIT Press, Cambridge MA.

[9] Hall, Melinda C. (2019) Critical Disability Theory. The Stanford Encyclopedia of Philosophy (Winter 2019 Edition). Available from http://plato.stanford.edu/archives/win2019/entries/disability-critical/ ( 27 August 2021 )

[10] S Hareli, K Kafetsios, & U Hess (2015) A Cross-cultural Study on Emotion Expression and the Learning of Social Norms. Frontiers in psychology, 6, 1501.

[11] A Howard, C Zhang, & E Horvitz (2017, March). Addressing bias in machine learning algorithms: A pilot study on emotion recognition for intelligent systems. 2017 IEEE Workshop on Systems. University of Oxford Connected Life 2018 – Conference Proceedings, 2334.

[12] M. H Immordino-Yang, X. F Yang, & H Damasio (2016) Cultural Modes of Expressing Emotions Influence How Emotions are Experienced. Emotion, 16(7), 1033.

[13] T Karras, M Aittala, J Hellsten, S Laine, J Lehtinen, & T Aila (2020). Training Generative Adversarial Networks with Limited Data. In IEEE Conference on Neural Information Processing Systems.

[14] N Kordzadeh, & M Ghasemaghaei (2021) Algorithmic Bias: Review, Synthesis, and Future Research Directions. European Journal of Information Systems, 1–22.

[15] P. J Lang, M. M Bradley, & B. N Cuthbert (1997) International Affective Picture System (IAPS): Technical Manual and Affective Ratings. NIMH Center for the Study of Emotion and Attention, 1(39-58), 3.

[16] M Levin, & T Siebers (2010) The Art of Disability: An Interview with Tobin Siebers. Disability Studies Quarterly, 30(2).

[17] A Maffei, & A Angrilli (2019) E-MOVIE-Experimental Movies for Induction Of Emotions In Neuroscience: An Innovative Film Database With Normative Data And Sex Differences. Plos One, 14(10).

[18] L Manovich (2017) Automating aesthetics: Artificial intelligence and image culture. Flash Art International, 316, 1–10.

[19] J McCormack, & d’M Inverno (2014). On the future of computers and creativity.Proceedings of AISB 2014: Symposium on Computational Creativity, London, 1 – 4 April 2014 .

[20] J McCormack, O Bown, A Dorin, J McCabe, G Monro, & M Whitelaw (2014) Ten Questions Concerning Generative Computer Art. Leonardo, 47(2), 135–141.

[21] M McLuhan, & Q Fiore (1967) The Medium Is the Message. New York, 123, 126–128.

[22] S Mohammad, & S Kiritchenko (2018, May). Wikiart emotions: An annotated dataset of emotions evoked by art. Proceedings of the eleventh international conference on language resources and evaluation LREC 2018.

[23] S Ogolla & A Gupta (2018). Inclusive Design – Methods to Ensure A High Degree Of Participation. In: Artificial Intelligence (AI)

[24] R. W Picard (1995). Affective Computing. In M.I.T Media Laboratory Perceptual Computing Section Technical Report No. 321.

[25] R. W Picard (2000). Affective Computing for HCI. HCI, 1, 829–833.

[26] Random Quark (2017). The Art of Feeling. Available from : http://randomquark.com/case-studies/mindswarms.html/ ( 27 August 2021 )

[27] P Riccio Design of a Brain-Computer Interface to Translate Emotional States Into Original Paintings. Diss. Politecnico di Torino, 2021.

[28] J. A Russell (1980). A Circumplex Model of Affect. Journal Of Personality and Social Psychology, 39(6), 1161.

[29] S Salevati, & S DiPaola (2015). A creative artificial intelligence system to investigate user experience, affect, emotion and creativity. Electronic Visualisation and the Arts (EVA 2015), 140–147.

[30] J Salminen, S. G Jung, & B. J Jansen (2019, May). Detecting demographic bias in automatically generated personas. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 1–6.

[31] A Schaefer, F Nils, X Sanchez, & P Philippot (2010). Assessing The Effectiveness of a Large Database of Emotion-Eliciting Films: A New Tool for Emotion Researchers. Cognition And Emotion, 24(7), 1153–1172.

[32] K Sherwood (2019). Out of the Blue: Art, Disability, and Yelling. In Contemporary Art and Disability Studies (pp. 213–224). Routledge.

[33] T Siebers (2005). Disability Aesthetics. PMLA/Publications of the Modern Language Association of America, 120(2), 542–546.

[34] P. K Solvang (2018). Between Art Therapy and Disability Aesthetics: A Sociological Approach for Understanding the Intersection Between Art Practice and Disability Discourse. Disability & Society, 33(2), 238–253.

[35] L Suchman, & J Weber (2016). Human-Machine Autonomies. Autonomous weapons systems: Law, Ethics, Policy, 75–102.

[36] W Sun, & Z Chen (2020). Learned Image Downscaling for Upscaling Using Content Adaptive Resampler. IEEE Transactions on Image Processing, 29, 4027–4040.

[37] N Watson, & S Vehmas (2019). Disability Studies: Into the multidisciplinary future. N Watson, S Vehmas (Eds.) Routledge Handbook of Disability Studies. Routledge, London.

[38] A Wierzbicka (1999). Emotions across languages and cultures: Diversity and universals. Cambridge University Press, Cambridge

[39] W Yang, K Makita, T Nakao, N Kanayama, M. G Machizawa, T Sasaoka, A Sugata, R Kobayashi, R Hiramoto, S Yamawaki, M Iwanaga, & M Miyatani (2018). Affective Auditory Stimulus Database: An Expanded Version of the International Affective Digitized Sounds (IADS-E). Behavior Research Methods, 50(4), 1415–1429.

[40] W. L Zheng, W Liu, Y Lu, B. L Lu, & A Cichocki (2018). Emotionmeter: A Multimodal Framework for Recognizing Human Emotions. IEEE Transactions on Cybernetics, 49(3), 1110–1122.

[41] P Zhong, D Wang, & C Miao (2020). EEG-based Emotion Recognition Using Regularized Graph Neural Networks. IEEE Transactions on Affective Computing, 1.

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65