Abstract
In this case of excellence, we present the design of a site-specific virtual heritage experience for a bomb shelter dating from the Spanish Civil War, known as Refugi 307. In order to maximize impact, we followed a co-design approach to analyse the project requirements for an educational experience for primary school students. In this way we included in the design the different needs and viewpoints of the involved stakeholders (students, teachers, curators, guides and interaction designers). The prototype was based on a novel Augmented Reality interaction paradigm, called World as Support. The goal was to complement the current guided visit in two ways: by linking physical evidence with historical events; and by fostering understanding of historical contents through emotional engagement and critical thinking. The results showed benefits of using digital augmentation and collaborative activities based on embodied exploration to enhance the educational experience.
Keywords
Augmented Reality, World-as-Support interaction paradigm, co-design strategies, Virtual Heritage
Cultural heritage sites play a crucial role in how people understand history and culture. According to Betsworth et al. (2014), they have “the potential to teach us important lessons, such as where we came from and subsequently, the people it has made us today”. While interactive experiences have been commonly used in these contexts to complement learning concepts (Flynn, 2013; Roussou et. al., 2015; Sakr et al., 2016), in recent years, a wide range of screen-based technology approaches have been explored in museums (Roussou et. al., 2015) and sites (Sakr et al., 2016) related to Cultural Heritage, as it is believed the interaction with multimedia content allows visitors to obtain a better understanding of past cultures (Müller et al., 2016).
Cultural Heritage places acquire their importance and meaning through situatedness; i.e., interpretation is possible because the visitor is physically present on site. However, in the currently dominant Window-on-the-World interaction paradigm (WoW), screen-based interfaces, such as smartphones or tablets, tend to draw the user’s attention away from the physical space onto a framed window, which tends to isolate him/her and to provide an individual experience (Müller et al., 2016). On the other hand, the design of interactive experiences for archaeological sites (Schaper et al., 2017) is particularly challenging, as very often these spaces cannot be modified by adding physical objects or installations. In such cases, visits are often complemented by guided tours and digital audio guides to direct visitors’ attention towards aspects that are not necessarily obvious without further explanations.
The goal of this research is to address the challenges of archaeological sites by exploring the educational potential of a recently emerging interaction paradigm called World-as-Support (WaS), which is based on projective Augmented Reality (AR) (Malinverni et al., 2017).
The World-as-Support interaction paradigm
The WaS paradigm is based on projective AR; i.e., augmentation is achieved by projecting the digital content onto the physical world surrounding the user via a handheld device. This portable system,
based on a mobile device with computer vision capabilities and a pico-projector, dynamically recognizes the environment (e.g.. topography, objects, users’ gestures and motion) and projects the digital information directly onto it. This allows interaction designers to take advantage of the benefits of Reality-Based Interaction, such as environment awareness and social awareness (Jacob et al., 2008). Moreover, as users act within the physical world to interact with digital content (Dourish, 2001), it unfolds the potential of bodily and tangible interaction, such as tangible manipulation, spatial interaction, and embodied facilitation (Hornecker and Buur, 2006).
We have conducted preliminary studies comparing the affordances of the WaS and the WoW paradigms in an educational environment (Malinverni et al., 2017). The results have shown that in the context of a storytelling application for primary students, the paradigm has the potential to enhance students’ learning experience in multiple ways: by supporting environmental awareness and context awareness, and by shaping the social relationships between users (Malinverni et al., 2017).
In the light of these promising results, we argue that the WaS paradigm should be applied to archaeological sites, where it would address three fundamental issues. Firstly, as the projection avoids the risks of users focusing only on the on-screen information, it provides a balanced addition of historical contents without disrupting the immersive experience of being on site. Secondly, as it allows multi-user engagement and collaborative tasks, it can help reveal new layers of the experience and promote different viewpoints. This shared action of meaning construction allows visitors to obtain a holistic understanding of the historical context. Finally, , as this paradigm allows users to interact with digital content directly in the physical world, , it can help deploy a more multi-sensory, embodied approach to the past, by which visitors can establish emotional anchors and strengthen their empathy with other people.
Case Study
To test these hypotheses, we carried out a study in the context of Refugi 307. This urban site is one of the 1,402 bomb shelters built in Barcelona by civilians during the Spanish Civil War, and was aimed at protecting the population from the many raids of the Italian and German aviation. This bomb shelter is nowadays part of the History Museum of the Barcelona (MUHBA) and can be visited by means of guided visits (figure1). Hence, the main goal of our project was to explore new ways to complement the current guided visit with an educational virtual heritage experience.
Figure 1. A school class following the guided visit and using the WaS system inside the Spanish Civil War bomb shelter Refugi 307.
Sample and procedure
To analyse the different aspects of the user requirements and to include the multiple needs and viewpoints of the involved stakeholders, we conducted an ethnographic study of the guided visit, as well as several co-design workshops with 40 students (girls = 18; boys = 22; mean age = 10.78 years old) and three teachers from a local school. We also interviewed a curator, a guide, and a museum educator. Some of the activities designed included the visualization of a bomb dropped by an airplane, the transportation of an injured person to the infirmary on a stretcher, building an arch that supports the tunnel, and listening to recreations of children testimonials recounting their experiences in the shelter.
Figure 5: Example of a collaborative activity in which two users project and transport a digital version of a stretcher to the original physical space of an infirmary
Finally, to evaluate the education potential of the WaS interaction paradigm, we tested (figure 1) a mid-tech prototype built during the previous workshops with a total of 20 children (girls = 11; boys = 9; mean age = 9.95 years old).
Methodology
The goal of the evaluation was to better understand stakeholders’ meaning-making of the educational experience in the bomb shelter by including their contributions from a perspective that went beyond the limits of verbal language. Due to children’s limited linguistic competence, it is often difficult for them to express their thoughts and explain ideas adequately. In this case, multimodal analysis (Kress, 2010; Malinverni et al., 2016; Sakr et al., 2016; Van Mechelen et al., 2017) has shown to effectively inform the design process (Malinverni et al., 2016; Van Mechelen et al., 2017). Multimodality is an interdisciplinary approach, derived from socio-semiotics, which aims to understand how people communicate and represent meaning in different forms (Price and Jewitt, 2013).
By collecting, analyzing and interpreting the multiple resources that children employed during the activities (e.g. body postures, spatial interaction, facial expressions, gaze, verbal expressions, drawings, etc.) we obtained useful insights into how children express their mental models in relation to the educational context through their body and person-environment interaction. Video- and audio recordings, annotations, and children’s contributions during the co-design workshops were transcribed into a descriptive format and analyzed using the software NVivo 11 and a grounded theory approach. As a result we were able to understand how in situ activities during the guided visit influenced children’s meaning-making processes.
Results and discussion
Our findings indicate the benefits of complementing the learning experience during the guided visit with supporting activities that involve digital augmentation of the physical space and that encourage embodied explorations such as spatial interaction, tangible manipulation and collaborative tasks.
Employing the WaS system allowed children to visualize missing objects in the physical space and therefore to illustrate certain aspects of life within the bomb shelter. The results show how these projections helped children better understand learning contents that were difficult to imagine. Furthermore, this feature also allowed pupils to compare different contents at the same time, and the results point towards the potential of WaS for supporting children’s capability of perspective-taking (Ackermann, 1996).
On the other hand, we found evidence that activities involving embodied exploration do enhance children’s understanding about topics requiring emotional engagement (Sakr et al., 2016), critical thinking (Rowan et al., 2016), and the notion of collaboration (Stanton et al., 2001). However, we concluded that due to the limited functionality of the current version of the prototype, this feature is still not fully explored. Previous studies (Malinverni and Pares, 2014) have demonstrated that embodied interaction (Dourish, 2001) can support learning of abstract concepts. We envision, through an improvement of the functionality of the prototype, to foster students’ understanding of underlying socio-cultural meanings such as the notion of identity (Smith et al., 2011) and solidarity. We believe that particularly activities building on collaborative learning (Doise et al., 1975; Malinverni and Pares, 2015; Nelson, 1994) and a shared construction of meaning (Ackermann, 2004) can promote a better understanding of these concepts.
Conclusions
This case study contributes to the body of advanced interfaces in cultural heritage by presenting and evaluating an educational experience based on the World as Support interaction paradigm for primary school children in the context of a Cultural Heritage site. Our study indicates a first set of benefits of using digital augmentations and participative activities based on embodied exploration. On the one hand, it allowed children to explore the physical environment in meaningful ways and to construct meaning by discovering new layers of the educational experience. On the other hand, the enactment of specific situations allowed the students to more directly experience historical content during the visit that required emotional engagement, critical thinking and collaborative learning. In future studies, we will explore further potentials of this approach with an advanced prototype.
References
Ackermann, E.K., 2004. Constructing Knowledge and Transforming the World, A learning zone of one’s own: Sharing representations and flow in collaborative learning environments. IOS Press.
Ackermann, E.K., 1996. Perspective-taking and object construction: two keys to learning. Constr. Pract. Des. Thinking, Learn. a Digit. World 25–37.
Betsworth, L., Bowen, H., Robinson, S., Jones, M., 2014. Performative technologies for heritage site regeneration. Pers. Ubiquitous Comput. 18, 1631–1650. doi:10.1007/s00779-014-0766-3
Doise, W., Mugny, G., Perret-Clermont, A.-N., 1975. Social Interaction and the Development of Cognitive Operations. Eur. J. Soc. Psychol. 5, 367–383. doi:10.1002/ejsp.2420050309
Dourish, P., 2001. Where the Action Is: The Foundations of Embodied Interaction. MIT press. doi:10.1162/leon.2003.36.5.412
Flynn, B., 2013. v-Embodiment for cultural heritage, in: Digital Heritage International Congress. IEEE, Marseille, France, pp. 347–354.
Hornecker, E., Buur, J., 2006. Getting a grip on tangible interaction: a framework on physical space and social interaction., in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 437–446.
Jacob, R.J.K., Girouard, A., Hirshfield, L.M., Horn, M.S., Shaer, O., Solovey, E.T., Zigelbaum, J., 2008. Reality-based interaction: a framework for post-WIMP interfaces., in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp. 201–210.
Kress, G., 2010. Multimodality. A social semiotic approach to contemporary communication. Routledge, London, UK.
Malinverni, L., Pares, N., 2014. Learning of Abstract Concepts through Embodied Interaction: a 10 years systematic review of Full Body Interaction Learning Environments. J. Educ. Technol. Soc.
Malinverni, L., Pares, N., 2015. The Medium Matters: the Impact of Full-Body Interaction on the Socio-Affective Aspects of Collaboration, in: IDC ’15 Proceedings of the 2015 Conference on Interaction Design and Children. ACM, New York, NY, USA. doi:10.1145/2771839.2771849
Malinverni, L., Maya, J., Schaper, M.-M., Pares, N., 2017. The World-as-Support: Embodied Exploration, Understanding and Meaning-Making of the Augmented World, in: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17). ACM, New York, NY, USA.
Malinverni, L., Mora-Guiard, J., Pares, N., 2016. Towards Methods for Evaluating and Communicating Participatory Design: a Multimodal Approach. Int. J. Hum. Comput. Stud. Manuscript submitted for publication. doi:10.1016/j.ijhcs.2016.03.004
Nelson, C.E., 1994. Critical thinking and collaborative learning. New Dir. Teach. Learn. 1994, 45–58. doi:10.1002/tl.37219945907
Price, S., Jewitt, C., 2013. Interview Approaches to Researching Embodiment 2907–2910.
Müller, J., Rädle, R., and Reiterer, H. 2016. Virtual Objects as Spatial Cues in Collaborative Mixed Reality Environments: How They Shape Communication Behavior and User Task Load. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16)., ACM, New York, NY, USA, 1245–1249. http://doi.org/http://dx.doi.org/10.1145/2858036.2858043
Rowan, L., Townend, G., Beavis, C., Kelly, L., Fletcher, J., 2016. Museums, Games, and Historical Imagination : Student Responses To a Games-Based Experience At the Australian National Maritime Museum. Digit. Educ. Cult. 8, 1–21.
Roussou, M., Pujol, L., Katifori, A., Chrysanthi, A., Perry, S., and Vayanou, M. 2015. The museum as digital storyteller: Collaborative participatory creation of interactive digital experiences. MW2015: Museums and the Web 2015. Retrieved November 19, 2015 from http://mw2015.museumsandtheweb.com/ paper/the-museum-as-digital-storyteller-collaborative-participatory-creation-of-interactive-digital-experiences/
Sakr, M., Jewitt, C., Price, S., 2016. Mobile Experiences of Historical Place: A Multimodal
Analysis of Emotional Engagement. J. Learn. Sci. 25, 51–92.
doi:10.1080/10508406.2015.1115761
Schaper, M.-M., Santos, M., Malinverni, L., Pares, N., 2017. Towards the Design of a Virtual Heritage Experience based on the World-as-Support Interaction Paradigm, in: In Proceedings of 35th the CHI Conference on Human Factors in Computing Systems (CHI ’17).
Stanton, D., O’Malley, C., Huing, K., Fraser, M., Benford, S., 2003. Situating Historical Events Through Mixed Reality, in: Designing for Change in Networked Learning Environments. Springer Netherlands, Dordrecht, pp. 293–302. doi:10.1007/978-94-017-0195-2_37
Van Mechelen, M., Derboven, J., Laenen, A., Willems, B., Geerts, D., Vanden Abeele, V., 2017. The GLID method: Moving from design features to underlying values in co-design. Int. J. Hum. Comput. Stud. 97, 116–128. doi:10.1016/j.ijhcs.2016.09.005
Author
Schaper, M., Pujol, L., Pares, N. (Cognitive Media Technologies Group, Universitat Pompeu Fabra) - Thematic Area 5