A Design Space for Memory Augmentation Technologies

The pervasive nature of display technologies can enable novel ever-accessible memory aids to address deterioration caused by ageing and cognitive decline. To date, however, memory has largely been treated as a single-unit, and there has been little formal consideration of how to select the most appropriate technology for a given intervention. We build on existing domain knowledge from neuroscience and psychology to suggest a novel design space with two axes: processing level, and display modality. In particular, we consider how augmentations might intervene at a biological, cognitive or meta-cognitive level using head-mounted (private) displays, small-scale (personal) displays, larger public and semi-public displays, and with technology that bypasses the visual channels entirely (e.g. through neural stimulation or non-visual senses). We then provide examples of potential studies to explore these design areas, and discuss future directions this approach to memory augmentation may take. Consideration is also given to the ethics of memory augmentation.

Content

Author and article information

Contributors

Madeleine Steeds

Sarah Clinch

Conference

Publication date: July 2021

Publication date (Print): July 2021

Pages: 76-81

Affiliations

[0001]University of Manchester

Manchester, UK

Article

DOI: 10.14236/ewic/HCI2021.6

SO-VID: 1cedff5e-a885-4477-a9ba-5604ab5f570f

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: 34th British HCI Conference

Conference acronym: HCI2021

Conference number: 34

Conference location: London, UK

Conference date: 20th - 21st July 2021

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Post-pandemic HCI – Living Digitally

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2021.6

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

L. Bäckman, L. Nyberg, U. Lindenberger, S. C. Li, & L. Farde (2006). The correlative triad among aging, dopamine, and cognition: current status and future prospects. Neuroscience & Biobehavioral Reviews, 30(6), 791-807.
Y. Chen and G. J. Jones (2010). Augmenting human memory using personal lifelogs. In Proceedings of the 1st augmented human international conference, New York, NY, USA, pp. 24. ACM.
N. Davies, A. Friday, S. Clinch, C. Sas, M. Langheinrich, G. Ward, and A. Schmidt (2015). Security and privacy implications of pervasive memory augmentation. IEEE Pervasive Computing 14(1), 44–53.
T. Dingler, C. Giebler, U. Kunze, T. Wundefinedrtele, N. Henze, and A. Schmidt (2016). Memory displays: Investigating the effects of learning in the periphery. In Proceedings of the 5th ACM International Symposium on Pervasive Displays, PerDis ’16, New York, NY, USA, pp. 118–123. Association for Computing Machinery.
R. Győrödi, C. Győrödi, G. Borha, M. Burtic, L. Pal, and J. Ferenczi (2015). Acquaintance reminder using google glass. In 2015 13th International Conference on Engineering of Modern Electric Systems (EMES), pp. 1–4.
C. Handra, O. A. Coman, L. Coman, T. Enache, S. Stoleru, A.-M. Sorescu, I. Ghita, and I. Fulga ˘ (2019). The connection between different neurotransmitters involved in cognitive processes. FARMACIA 67(2), 193–201.
R. E. Hampson, D. Song, B. S. Robinson, D. Fetterhoff, A. S. Dakos, B. M. Roeder, X. She, R. T. Wicks, M. R. Witcher, D. E. Couture, et al. (2018). Developing a hippocampal neural prosthetic to facilitate human memory encoding and recall. Journal of neural engineering 15(3), 036014.
M. Harvey, M. Langheinrich, and G. Ward (2016). Remembering through lifelogging: A survey of human memory augmentation. Pervasive and Mobile Computing 27, 14–26.
S. Hodges, E. Berry, and K. Wood (2011). Sensecam: A wearable camera that stimulates and rehabilitates autobiographical memory. Memory 19(7), 685–696.
S. Hodges, L. Williams, E. Berry, S. Izadi, J. Srinivasan, A. Butler, G. Smyth, N. Kapur, and K. Wood (2006). Sensecam: A retrospective memory aid. In International Conference on Ubiquitous Computing, pp. 177–193. Springer.
M. Iwamura, K. Kunze, Y. Kato, Y. Utsumi, and K. Kise (2014). Haven’t we met before?: a realistic memory assistance system to remind you of the person in front of you. In Proceedings of the 5th Augmented Human International Conference, New York, NY, USA, pp. 32. ACM.
S. Kuznetsov, A. K. Dey, and S. E. Hudson (2009). The effectiveness of haptic cues as an assistive technology for human memory. In International Conference on Pervasive Computing, pp. 168–175. Springer.
H. V. Le, S. Clinch, C. Sas, T. Dingler, N. Henze, and N. Davies (2016). Impact of video summary viewing on episodic memory recall: Design guidelines for video summarizations. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, pp. 4793–4805. ACM.
C. R. Madan (2014). Augmented memory: a survey of the approaches to remembering more. Frontiers in systems neuroscience 8, 30.
M. Mikusz, S. Clinch, P. Shaw, N. Davies, and P. Nurmi (2018). Using pervasive displays to aid student recall -reflections on a campus-wide trial. In Proceedings of the 7th ACM International Symposium on Pervasive Displays, PerDis ’18, New York, NY, USA. Association for Computing Machinery.
J. Muller, F. Alt, D. Michelis, and A. Schmidt (2010). ¨ Requirements and design space for interactive public displays. In Proceedings of the 18th ACM international conference on Multimedia, New York, NY, USA, pp. 1285–1294. ACM.
N. Reggente, J. K. Essoe, H. Y. Baek, and J. Rissman (2020). The method of loci in virtual reality: explicit binding of objects to spatial contexts enhances subsequent memory recall. Journal of Cognitive Enhancement 4(1), 12–30.
B. J. Rhodes (1997). The wearable remembrance agent: A system for augmented memory. Personal Technologies 1(4), 218–224.
A. Schmidt (2017). Augmenting human intellect and amplifying perception and cognition. IEEE Pervasive Computing 16(1), 6–10.
C. Seim, J. Chandler, K. DesPortes, S. Dhingra, M. Park, and T. Starner (2014). Passive haptic learning of braille typing. In Proceedings of the 2014 ACM International Symposium on Wearable Computers, New York, NY, USA, pp. 111–118. ACM.
C. Seim, T. Estes, and T. Starner (2015). Towards passive haptic learning of piano songs. In Proceedings of the 2015 World Haptics Conference, pp. 445–450. IEEE. M. Solis (2017). Committing to memory: Memory prosthetics show promise in helping those with neurodegenerative disorders. IEEE pulse 8(1), 33–37.
Y. Shu, Y.-Z. Huang, S.-H. Chang, and M.-Y. Chen (2019). Do virtual reality head-mounted displays make a difference? A comparison of presence and self-efficacy between head-mounted displays and desktop computer-facilitated virtual environments. Virtual Reality 23(4), 437–446.
M. Solis (2017). Committing to memory: Memory prosthetics show promise in helping those with neurodegenerative disorders. IEEE pulse 8(1), 33–37.
F. Yang, J. Qian, J. Novotny, D. Badre, C. Jackson, and D. Laidlaw (2020). A virtual reality memory palace variant aids knowledge retrieval from scholarly articles. IEEE Transactions on Visualization and Computer Graphics.

Comments

Comment on this article

[1] L. Bäckman, L. Nyberg, U. Lindenberger, S. C. Li, & L. Farde (2006). The correlative triad among aging, dopamine, and cognition: current status and future prospects. Neuroscience & Biobehavioral Reviews, 30(6), 791-807.

[2] Y. Chen and G. J. Jones (2010). Augmenting human memory using personal lifelogs. In Proceedings of the 1st augmented human international conference, New York, NY, USA, pp. 24. ACM.

[3] N. Davies, A. Friday, S. Clinch, C. Sas, M. Langheinrich, G. Ward, and A. Schmidt (2015). Security and privacy implications of pervasive memory augmentation. IEEE Pervasive Computing 14(1), 44–53.

[4] T. Dingler, C. Giebler, U. Kunze, T. Wundefinedrtele, N. Henze, and A. Schmidt (2016). Memory displays: Investigating the effects of learning in the periphery. In Proceedings of the 5th ACM International Symposium on Pervasive Displays, PerDis ’16, New York, NY, USA, pp. 118–123. Association for Computing Machinery.

[5] R. Győrödi, C. Győrödi, G. Borha, M. Burtic, L. Pal, and J. Ferenczi (2015). Acquaintance reminder using google glass. In 2015 13th International Conference on Engineering of Modern Electric Systems (EMES), pp. 1–4.

[6] C. Handra, O. A. Coman, L. Coman, T. Enache, S. Stoleru, A.-M. Sorescu, I. Ghita, and I. Fulga ˘ (2019). The connection between different neurotransmitters involved in cognitive processes. FARMACIA 67(2), 193–201.

[7] R. E. Hampson, D. Song, B. S. Robinson, D. Fetterhoff, A. S. Dakos, B. M. Roeder, X. She, R. T. Wicks, M. R. Witcher, D. E. Couture, et al. (2018). Developing a hippocampal neural prosthetic to facilitate human memory encoding and recall. Journal of neural engineering 15(3), 036014.

[8] M. Harvey, M. Langheinrich, and G. Ward (2016). Remembering through lifelogging: A survey of human memory augmentation. Pervasive and Mobile Computing 27, 14–26.

[9] S. Hodges, E. Berry, and K. Wood (2011). Sensecam: A wearable camera that stimulates and rehabilitates autobiographical memory. Memory 19(7), 685–696.

[10] S. Hodges, L. Williams, E. Berry, S. Izadi, J. Srinivasan, A. Butler, G. Smyth, N. Kapur, and K. Wood (2006). Sensecam: A retrospective memory aid. In International Conference on Ubiquitous Computing, pp. 177–193. Springer.

[11] M. Iwamura, K. Kunze, Y. Kato, Y. Utsumi, and K. Kise (2014). Haven’t we met before?: a realistic memory assistance system to remind you of the person in front of you. In Proceedings of the 5th Augmented Human International Conference, New York, NY, USA, pp. 32. ACM.

[12] S. Kuznetsov, A. K. Dey, and S. E. Hudson (2009). The effectiveness of haptic cues as an assistive technology for human memory. In International Conference on Pervasive Computing, pp. 168–175. Springer.

[13] H. V. Le, S. Clinch, C. Sas, T. Dingler, N. Henze, and N. Davies (2016). Impact of video summary viewing on episodic memory recall: Design guidelines for video summarizations. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, pp. 4793–4805. ACM.

[14] C. R. Madan (2014). Augmented memory: a survey of the approaches to remembering more. Frontiers in systems neuroscience 8, 30.

[15] M. Mikusz, S. Clinch, P. Shaw, N. Davies, and P. Nurmi (2018). Using pervasive displays to aid student recall -reflections on a campus-wide trial. In Proceedings of the 7th ACM International Symposium on Pervasive Displays, PerDis ’18, New York, NY, USA. Association for Computing Machinery.

[16] J. Muller, F. Alt, D. Michelis, and A. Schmidt (2010). ¨ Requirements and design space for interactive public displays. In Proceedings of the 18th ACM international conference on Multimedia, New York, NY, USA, pp. 1285–1294. ACM.

[17] N. Reggente, J. K. Essoe, H. Y. Baek, and J. Rissman (2020). The method of loci in virtual reality: explicit binding of objects to spatial contexts enhances subsequent memory recall. Journal of Cognitive Enhancement 4(1), 12–30.

[18] B. J. Rhodes (1997). The wearable remembrance agent: A system for augmented memory. Personal Technologies 1(4), 218–224.

[19] A. Schmidt (2017). Augmenting human intellect and amplifying perception and cognition. IEEE Pervasive Computing 16(1), 6–10.

[20] C. Seim, J. Chandler, K. DesPortes, S. Dhingra, M. Park, and T. Starner (2014). Passive haptic learning of braille typing. In Proceedings of the 2014 ACM International Symposium on Wearable Computers, New York, NY, USA, pp. 111–118. ACM.

[21] C. Seim, T. Estes, and T. Starner (2015). Towards passive haptic learning of piano songs. In Proceedings of the 2015 World Haptics Conference, pp. 445–450. IEEE. M. Solis (2017). Committing to memory: Memory prosthetics show promise in helping those with neurodegenerative disorders. IEEE pulse 8(1), 33–37.

[22] Y. Shu, Y.-Z. Huang, S.-H. Chang, and M.-Y. Chen (2019). Do virtual reality head-mounted displays make a difference? A comparison of presence and self-efficacy between head-mounted displays and desktop computer-facilitated virtual environments. Virtual Reality 23(4), 437–446.

[23] M. Solis (2017). Committing to memory: Memory prosthetics show promise in helping those with neurodegenerative disorders. IEEE pulse 8(1), 33–37.

[24] F. Yang, J. Qian, J. Novotny, D. Badre, C. Jackson, and D. Laidlaw (2020). A virtual reality memory palace variant aids knowledge retrieval from scholarly articles. IEEE Transactions on Visualization and Computer Graphics.

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65