2,004
views
0
recommends
+1 Recommend
1 collections
    2
    shares

      Studying business & IT? Drive your professional career forwards with BCS books - for a 20% discount click here: shop.bcs.org

      scite_
       
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      AI and Cultural Heritage Image Collections: Opportunities and challenges

      Published
      proceedings-article
      ,
      Proceedings of EVA London 2021 (EVA 2021)
      AI and the Arts: Artificial Imagination
      5th July – 9th July 2021
      Image classification, Deep learning, Convolutional neural networks, Cultural heritage, Digital humanities
      Bookmark

            Abstract

            This paper contributes to the discussion about the opportunities and challenges of applying computer vision and machine learning to archival image collections of significant cultural heritage value. We explore these questions from an institutional perspective. Our case study is a pilot project developed at Dumbarton Oaks, a research institute and library, museum, and historic garden affiliated with Harvard University and located in Washington, DC. The project focused on a collection of 10,000 images of Syrian monuments in the institution’s Image Collections and Fieldwork Archives (ICFA). Drawing on that project, as well as the broader landscape of AI-based categorisation efforts in the fields of art and architecture, we will share our insights on the potential of AI to facilitate and enhance archival image access and recording. Many of the Syrian sites in the Dumbarton Oaks collection have been inaccessible to researchers and the public for over a decade and/or have been damaged or destroyed. The pilot project, undertaken in 2019-2020, was a collaboration between Dumbarton Oaks; a commercial tech partner, ArthurAI Inc.; and a computer science research team from the University of Maryland. For Dumbarton Oaks, the primary goal was to explore whether AI can improve the speed and efficiency of sharing collections and allow for more sophisticated curation by subject experts who, thanks to automation, would be relieved of the burden of rote processing. For the technology partners, the experimental value of the project lay in the availability of a collection that could be shared open access (no privacy or copyright issues) and was focused enough to yield a domain-specific training set. The methods and techniques explored included multi-label classification, multi-task classification, unsupervised image clustering, and explainability.

            REFERENCES

            1. , , and (2016). Visual Recognition of Ancient Inscriptions Using Convolutional Neural Network and Fisher Vector. Journal on Computing and Cultural Heritage, 9(4),21.

            2. , , and (2018) Leveraging Known Data for Missing Label Prediction in Cultural Heritage Context. Applied Sciences, 8(10), 1768.

            3. (2019) Exploring Art with Open Access and AI: What’s Next? https://www.metmuseum.org/blogs/now-at-themet/2019/met-microsoft-mit-exploring-art-open-access-ai-whats-next (retrieved 13 March 2021).

            4. (2020) Machine Learning + Libraries: A Report on the State of the Field. Library of Congress, Washington, D.C.

            5. , , , , and (2018) The Shape of Art History in the Eyes of the Machine. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, New Orleans.

            6. , , , and (2020) Detection of Disaster-Affected Cultural Heritage Sites from Social Media Images Using Deep Learning Techniques. Journal on Computing and Cultural Heritage, 13(3), 23.

            7. , , , , , , and (2021) MonuMAI: Dataset, Deep Learning Pipeline and Citizen Science Based App for Monumental Heritage Taxonomy and Classification. Neurocomputing, 420, pp. 266–80.

            8. , , , and (eds.) (2017). Deep Learning and Convolutional Neural Networks for Medical Image Computing: Precision Medicine, High Performance and Large-Scale Datasets. Springer International Publishing, Cham.

            9. , , and (2015) Deep Learning. Nature, 521 (7553), pp. 436-444.

            10. , , , and (2017) Classification of Architectural Heritage Images Using Deep Learning Techniques. Applied Sciences, 7, 992.

            11. , , , and (2020) Impact of Industry 4.0 in Architecture and Cultural Heritage: Artificial Intelligence and Semantic Web Technologies to Empower Interoperability and Data Usage. In and (eds.), Impact of Industry 4.0 on Architecture and Cultural Heritage. IGI Global, Hershey.

            12. (2019) Towards Deep Learning for Architecture: A Monument Recognition Mobile App. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (XLII-2/W9 2019), Bergamo, Italy, 6-8 February 2019, 551–556. Bergamo.

            13. , , and (2019). Two-Stage Deep Learning Approach to the Classification of Fine-Art Paintings. IEEE Access, 7, pp. 41770-41781.

            14. , ., , , and (2019) Classifying Digitized Art Type and Time Period. Proceedings of International Journal for Digital Art History (DSDAH 2018), London, UK, August 2018. London.

            Author and article information

            Contributors
            Conference
            July 2021
            July 2021
            : 193-198
            Affiliations
            [0001]Dumbarton Oaks, Harvard University

            1703 32 nd Street NW, Washington, DC, 20007, USA
            [0002]
            Article
            10.14236/ewic/EVA2021.33
            cb8d580b-aec2-4d0c-9751-90b3348473e8
            © Karterouli et al. Published by BCS Learning & Development Ltd. Proceedings of EVA London 2021, UK

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            Proceedings of EVA London 2021
            EVA 2021
            London
            5th July – 9th July 2021
            Electronic Workshops in Computing (eWiC)
            AI and the Arts: Artificial Imagination
            History
            Product

            1477-9358 BCS Learning & Development

            Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EVA2021.33
            Self URI (journal page): https://ewic.bcs.org/
            Categories
            Electronic Workshops in Computing

            Applied computer science,Computer science,Security & Cryptology,Graphics & Multimedia design,General computer science,Human-computer-interaction
            Cultural heritage,Deep learning,Digital humanities,Convolutional neural networks,Image classification

            Comments

            Comment on this article