The heterogeneity of the audience of cultural heritage institutions introduces numerous challenges to the delivery of the content. Considering that people differ in the way they perceive, process, and recall information and that their individual cognitive differences influence their experience, performance, and knowledge acquisition when performing cultural-heritage activities, the cognition factor should be considered as an important personalization factor within cultural-heritage contexts. To this end, in this case -study, we present a cognition-centered approach for delivering personalized cultural-heritage activities, tailored to the users’ cognitive characteristics. The approach supports both the pre-visit (e.g., a designer creates a cognition-centered personalized activity) and during-visit (e.g., a visitor performs an activity that is tailored to their unique cognitive characteristics) stages. The results of our recent studies provide evidence about the applicability, effectiveness, and efficiency of the proposed approach and underpin the added value of adopting cognition-centered personalization within digitized cultural-heritage interaction contexts.
cultural heritage, cognition, personalization, adaptation, user modeling
Cultural heritage has been a favored domain for personalization throughout the years, given the numerous reasons that necessitate the personalization, such as the heterogeneity of the visitors, the huge amount of information provided by the cultural-heritage institutions, and the diversity of the organizational resources of the institutions. These needs have driven the design of personalized cultural activities based on several factors, such as technological, institutional, and human factors. Focusing on the human factors, designing for them is challenging, as they are related to the individual visitors’ characteristics (e.g., goals, backgrounds, information processing styles). While the personalization of cultural activities considers varying human factors, such as preferences, interests, personality, and behavior, the current approaches barely take into consideration human cognition. Considering that cultural activities include information-processing tasks and that people with different cognitive characteristics follow different approaches when performing such tasks, which result in differences in their experience, performance, visiting style, and knowledge acquisition, human cognition is expected to be a significant personalization factor in cultural-heritage contexts. Therefore, there is a necessity of considering human cognition as a building block of a human-centered approach that will be used by cultural-heritage stakeholders (e.g., designers and content providers) to create personalized activities tailored to the visitors’ cognitive characteristics, aiming to enhance their overall visiting experience.
Our cognition-centered approach consists of three stages: a) the design of personalized cultural activities by cultural-heritage designers, b) the visitor modeling based on selected cognitive factors, and c) the configuration and delivery of the personalized cultural activity to the visitor. The first stage is performed before the visit while the other two stages are performed during the visit.
Design of personalized cultural activities
To design a personalized cultural activity, a cultural heritage designer could use our recently developed design tool. The tool supports design decisions through the various stages of the design cycle, which reflect in specific design functions, such as task definition, choice of representations, choice of methods, and definition of visualization, interaction, and distribution strategies. In turn, the aforementioned functions are supported by providing well-defined design principles, such as design heuristics and patterns.
Our cognition-centered approach  is based on three main factors: activity, objective, and cognition, which are interrelated. Activity factor contains information about the activity characteristics, such as type and mechanics; Objective factor contains information about the objective (e.g., enhance visit experience) of the cultural activity; Cognition factor contains information about the cognitive characteristics (e.g., cognitive styles, cognitive abilities) that can be used as the basis of the creation of adaptive cognition-centered interventions.
The design tool consists of three main components: a) the setup component, which is used by the designer to initialize the process and provide the selection criteria; b) the software interface, which provides the configuration element, which creates requests in the database, and the design principles and cognition factors, which meet the selection criteria; c) the design-space database, which consists of collections of interrelated data-sets, such as design principles and activity factors.
One of the main challenges of a cognition-centered approach for delivering personalized experiences is the user-modeling, considering that the elicitation of cognitive characteristics is an explicit and time-consuming activity which often requires human intervention (e.g., a facilitator assesses the responses of the individuals in a psychometric test and classifies them to a dimension of a cognitive style). That is even more difficult in activities that people engage within cultural heritage contents, who typically engage in the activity only once and who have limited time (e.g., a tourist visits an archaeological site for a first – and probably only one – time in their life and has time constraints regarding the visit duration).
To address this challenge, our approach uses interactive and eye-tracking data to model the visitors and classify them into user groups, based on their cognitive characteristics. The visitor modeling process is integrated within the cultural activity and is transparent to the visitor. No facilitator is needed as the assessment is performed independently by the visitor modeling component, which has been optimized, through machine learning processes, to provide the best fitted cognitive profile to the visitor dynamically and transparently in the very early stages of the cultural activity.
Configuration and delivery of the personalized cultural activity
In this stage, the activity version (derived during the first stage) that meets the visitor’s cognitive profile (derived during the second stage) is delivered to the visitor. Therefore, the configuration and delivery of the cognition-centered personalized cultural activity are based on rule-based and predictive techniques, which aim to adapt the content dynamically, based on a set of pre-defined rules-set (e.g., activity version, activity elements) while mapping the predicted visitor’s cognitive profile. This stage is based on technical details, which the reader can find in the recently published work of Raptis et al. . As an example, the rule for individuals with low working memory is to help them to produce longer fixations into critical areas of interest when performing a visual exploratory cultural activity (e.g., viewing a painting); hence, the personalized configuration for that type of cognitive profile would be to apply salience filters to the critical areas of interest, which will be activated when the user produces fixations longer than a pre-defined threshold.
A scenario that implements the discussed approach is:
- The designer initializes the design tool by selecting the activity characteristics, the objective of the activity, and the design stages or functions in which they will apply the derived principles;
- the selected features are sent to the configuration element;
- the configuration element requests the design principles, which meet the selection criteria, from a database;
- a software agent returns a set of the suggested design principles along with the corresponding cognitive factors;
- the designer receives the matched design principles (along with the cognition factors) and uses them to create new cultural activities or adjust existing ones, tailored to the cognitive characteristics of the visitors.
- The visitor receives the default version of the cultural activity;
- the user-modeling component assigns transparently a cognitive profile to the visitor, leveraging on interaction and/or eye-gaze data, in the very early stages of the activity;
- a software agent re-adjusts the activity characteristics (e.g., elements, mechanics) to meet the visitor’s cognitive profile;
- a cognition-centered personalized version of the activity is delivered to the visitor, aiming to enhance their experience.
For evaluating the proposed approach, we conducted various eye-tracking between-subjects user-studies covering different cognitive styles and different types of cultural activities. The results of our recent studies  underpin that:
- the cognition-centered personalization was effective, as it improved visitors’ gains towards activity objectives and mitigated imbalances between individuals with different cognitive characteristics;
- implicit cognition-centered user-modeling through transparent and in-run-time eye-gaze based elicitation mechanisms is feasible with high accuracy, sensitivity, and specificity during the early stages of visual goal-oriented and exploratory cultural-heritage activities;
- there is a need of adopting such approaches in cultural-heritage contexts, as an imbalance (e.g., towards knowledge acquisition) between individuals of different cognitive characteristics is observed in current design approaches.
In this case study, we presented our cognition-centered approach that is conceptualized on the triadic relationship among activity-objective-cognition and its influence on the visitors’ behavior when performing cultural activities. The cognition-centered approach aims to help both the visitors of a cultural heritage site, as they receive personalized activities, and the cultural-heritage stakeholders, such as designers, as they are supported to provide personalized activities. Finally, the open nature of the approach allows its application in other domains and enables its use for providing personalized solutions in other triadic relationships.
George E. Raptis, Christos A. Fidas, Christina Katsini, and Nikolaos M. Avouris. 2019. A cognition-centered personalization framework for cultural-heritage content. User Modeling and User-Adapted Interaction (13 Feb 2019). https://doi.org/10.1007/s11257-019-09226-7.
*** The featured image is a collage of Water vector created by rawpixel.com – www.freepik.com and Watercolor vector created by freepik – www.freepik.com images.
George E. Raptis, HCI Group, ECE Department, University of Patras, Greece
The discussed approach is part of George E. Raptis’ PhD project entitled “Study, Design, and Development of an Adaptive Framework Supporting Personalized Interaction with Cultural Heritage Applications taking into consideration Individual Differences in Perception and Information Processing” supervised by Prof. Nikolaos Avouris and advised by Asst. Professor Christos Fidas and Prof. George Lepouras.
For more information about the cognition-centered framework, read the recently published article by Raptis et al.  in User Modeling and User-Adapted Interaction (UMUAI) journal. For more information about the PhD project, visit the HCI Group website (https://hci.ece.upatras.gr) and the personal website of George E. Raptis (http://www.raptisg.com).