Social and Cognitive Factors in the Design and Evaluation of Multimedia Systems

Multimedia information retrieval systems design is currently considered a complex task since it requires an understanding of users specific way of searching, processing and using data conveyed by different media (texts, images, videos, sounds etc.). In this work we first discuss the state of the art of research in the field of textual and visual information retrieval approaching its problems from a user-centred perspective. We then propose a general framework derived from recent studies in social and cognitive ergonomics that should prove useful to guide the design and evaluation of user-system interaction in a multimedia context. We also provide some examples of social and cognitive factors affecting both the design and the evaluation of multimedia systems interfaces, discussing them with particular reference to the field of Digital Libraries. We conclude by stressing some points of strength of our approach and by suggesting new research issues to be further investigated.


Introduction
The multimedia revolution is posing some very new challenges to designers working at the development of advanced information retrieval systems.The complexity of the problem consists on the difficulty of finding effective data representation formalisms, cataloguing methods, retrieval algorithms and smart interfaces capable of supporting users in accessing the desired pieces of information in the context of their searching tasks.
During the last ten years it has been well accepted by the scientific community that a system success depends heavily on its capacity of matching users needs as well as their natural way of thinking and acting, even if reaching this goal is often the result of some good intuitions or handicraft capabilities on the part of designers and the outcome of an iterative and effortful design and evaluation process.
Focusing designers attention on users social, cognitive and perceptual abilities is becoming even more compulsory today when so little is known about people way of processing information conveyed by media different from texts as for example images, videos, music and graphics.
It has been suggested that to approach the design of multimedia systems some interdisciplinary knowledge should be applied, in particular that coming from studies in information science, information engineering, computer science, visual/mass communications and cognitive psychology [1].
In the following we will discuss some current issues related to the development of multimedia IR applications with special regard for the design of systems enabling the retrieval of images and textual information and their relevance for the field of Digital Libraries (DL).
We will then present a general framework derived from recent studies in cognitive ergonomics [2,3] which can provide some useful guidelines for the design and evaluation of multimedia systems interfaces, taking into account the most relevant social and cognitive factors affecting users interactive behaviour with information technology.We will also briefly discuss some problems related to the design of empirical studies and experiments to evaluate and enhance multimedia systems ability to cope with users needs and preferences at both a social and cognitive level of analysis.
We will conclude by emphasizing some points of strength of the framework proposed and by suggesting new research questions for future studies.

Current Issues in Multimedia Information Retrieval
The area of multimedia information retrieval is concerned with the development of different kind of applications whose main goal is that of enabling users to perform different types of information-seeking tasks.
The most common ones are: • search tasks, for example the retrieval of a specific document which satisfies a particular information need; • browsing tasks, for example exploring a wide collection of documents to get an idea of what a specific knowledge domain is about.
In both cases users can face many problems in accessing the multimedia objects they need if information are not properly represented in the system and if the type of communication performed at the interface level doesn't help the searching process.
Within the community interested in the enhancement of DL there's recently a growing debate about the importance of defining some good methods and sharable tools to help designers in managing large data collections deciding, in particular, how to provide their appropriate forms of representation.
During the last ten years discussions have concentrated especially on the challenges introduced by Visual Information Retrieval (VIR), since this area poses some very new questions with respect to the more traditional way of representing and retrieving textual information [4].
A first and simpler type of solution proposed to represent and retrieve images has been that of describing or indexing their thematic contents using textual attributes, as for example words extracted from captions or other pieces of text situated physically close to them.
The most obvious drawbacks involved in this solution are the following: • the vocabolary problem, that is the mismatch between user queries and words representing images in the system, • the lack of a consistent and rich way of representing data contained in the image database so as to satisfy different kinds of requests posed by different categories of users.
An alternative approach consists in studying which is the most appropriate way of representing images in a database using content-based methods.In this case a decision must be taken on which features (that is image characteristics that can be used to describe its visual contents) are best candidates for representing images contents for a specific application.Then some automatic algorithms can be developed and applied to extract this data by processing images at the moment of their introduction in the database or at the time of query processing [5].
Automatic procedures are considered well adapted to speed up the cataloguing process for large document collections and also to preserve a consistent and objective method of representing image contents.Nevertheless the choice of which features to extract (shape, color, texture, or a combination of these) is not so straightforward; it should be tailored to the kind of material (images) to process and to the kind of queries users will pose.Recent studies have demonstrated how difficult it is to represent images belonging to arts and humanities collections with enough rich and expressive power to satisfy users retrieval needs [1].The problem doesn't consist only on the development of automatic features extraction methods and on deriving semantics (high-level domain concepts manifested in the image) through a human expert involvement.A much more complex task is that of decomposing images content (but the same is also true for other media like videos or sounds), from a perceptual and conceptual point of view, identifying the cognitive dimensions matching users typical IR needs.
Moreover we have to consider that automatic features extraction methods can provide data at a low level of abstraction, so even if they carry some advantages in speeding up the image processing task, they cannot be sufficient to solve all the cataloguing and data representation problems, since data at a higher level of abstraction are also necessary.
Another observation relevant for DL development is that content-based methods and algorithms for the processing and representation of images often adopt some ad-hoc solutions which may be appropriate for a specific multimedia application but may not fit the needs and styles of interaction of the different categories of users that a DL, for example, can have.
Sometimes they are not easily sharable or integrable with other text-based cataloguing methods and algorithms developed for other applications which represent different collections of documents.This is a still ananswered question for DL since for integration and interoperability purposes some well established and accepted standards are not available yet; for audiovisual material the development of a standard on the content-based description of multimedia data is at present studied in MPEG-7 [6; http://www.cselt.stet.it/mpeg].We also think that the development of standards must be supported by a more mature stage of research in multimedia information processing which includes not only a technology driven approach but also a deeper understanding of human cognitive abilities and characteristics of processing multimedia information.
Particularly relevant for DL interfaces design is to establish some new search models that can support users specific ways of accessing information material in a non textual form.In the field of image retrieval it has been noticed how visual thought is stimulated by images [1].For this reason it is important to enable users to start their searching tasks, for example, choosing among some initial sample images offered by the system, instead of being constrained to express their information need using a textual query.In [7] users can initially explore a large collection of images by looking at the distribution of images in a virtual space whose axes represent image features they themselves select.
Images selected by a user can constitute a helpful base to continue a searching task by performing query refinement, in particular through the exploitation of similarity search functions.These observations should be taken into account in designing search models and interfaces for the purpose of matching user IR strategies and of augmenting the points of access to the stream of data contained in multimedia collections of documents.
From a cognitive point of view we can observe that in VIR the data recognition and interpretation processes seem to be more important in comparison with the simple retrieval of specific answers in the system.A typical users expression that stresses this point is the saying 'I will know it when I see it', which demonstrates also the vagueness of their initial queries and the importance of browsing activity, at first, in accessing multimedia materials.So we think that interactivity should be enhanced as well as flexibility and adaptivity of interface functions to users task needs and preferred way of searching.Some lessons for approaching this kind of problems could be learned by analyzing the models and proposed solutions elaborated within the field of hypermedia systems development, since with the recent advent of World Wide Web [WWW] technology the decision of organizing multimedia collections of documents in hypertextual structures has imposed.For an overview and some examples of applications developed in this field see [8].
In concluding this section we want to highlight that in the next years some improvement should be made in the methods of representing, cataloguing and searching multimedia documents providing efficient tools to integrate the traditional text based retrieval techniques with the more advanced and innovative content-based algorithms.
We believe that to design some good and really supportive interfaces other complex issues should be considered, pertaining more specifically to a social and cognitive analysis level of user-system interaction.It is this important issue that we will address in the following.

A Reference Framework for Designing and Evaluating User-System Interaction in Multimedia Applications
The aim of approaching the task of designing a multimedia application from an interdisciplinary perspective consists in avoiding the risk of making choices driven just by what technology seems to offer at a particular time.Instead we think that a more complete and richer understanding of users interaction with multimedia applications can be reached taking into consideration both the potentialities and the constraints imposed by the environment context in which the system will be used.
The results of a series of studies carried out recently in the field of social and cognitive ergonomics can be very helpful to this purpose since they define a framework of questions at different levels of analysis that should guide the design process of such complex systems as multimedia IR applications.
In this work we will compare two recently developed frameworks proposed respectively in [2,3] see also [9] both emphasizing the consideration of social and cognitive factors in the design of complex systems.
In [2] the design of artifacts, such as information technology systems for example, is conceptualized by reference to a three level model of analysis.1.
Actors-environment interaction is considered at the first level of the model as the construction of social contexts; within a social context the decisions and actions of the different actors involved are influenced and, in turn, continually shape and change cultural models and social norms present at a certain time in the environment considered.This analysis perspective is strictly linked to recent studies in anthropology and cultural psychology [10,11].

2.
The second level of the model has its focus on actors interpretations of situations which are considered as the premises to actors definitions and elaborations of goals.Goals are a consequence of the matching between opportunities found in the environment and actors personal interests.This point represents a view emerged within the 'cognition in practice' approach in cognitive science [12,13].

3.
In the third level of the model the analysis is centred on actors local interaction with the environment, in particular with cognitive artifacts [14], such as software applications, through which users perform their tasks.
In [2] it is stressed that Human Computer Interaction studies have devoted much more attention to analysis at level 3 of the model, focusing on the understanding of user-system interaction through the interface functionalities.Some studies have also addressed the level of situations to understand actors representations of systems and users 'maps' guiding their choices or judgements of relevance about systems use for the fulfillment of their goals.There are very few studies concerned with issues at level 1 in the model, that make explicit the presence of specific social norms influencing designers and users mental models of contexts and the meanings they attribute to information technology tools as artifacts shaping and changing their daily activities.It should be interesting to understand more deeply such processes as actors negotiations of meanings and interpretations of situations at the interplay between, for example, different viewpoints typical of actors belonging to different professional groups.
The second framework we introduce here is the one elaborated in [3] from a tradition of studies in cognitive ergonomics, which have been applied in particular to the design and evaluation of IR systems.The framework emphasizes the importance of performing a work domain analysis and evaluation that "must serve to represent the characteristics of both the physical work environment and the situational interpretation of this environment by the actors involved, depending on their physical, perceptual properties and their skills, strategies and values" [3].The framework indicates several levels of work analysis and specifies that at each level designers or analysts choices "remove actions alternatives on users part defining behavior shaping constraints at progressively narrow envelopes" [3].
For the analysis of human-work interaction mediated by computer it is suggested to consider the work domain in terms of its means-ends space.This includes a representation of the landscape within which the work takes place; the goals, constraints and productive resources available to actors for designing their local activity, indipendent of particular situations and tasks.
We can compare this level of analysis with level 1 in the first model presented and can derive some relevant questions to ask for the decisions to be taken.Some examples are: does the system support cooperative work coordination, does it match policies for organizational and actors development ?Social elements such as norms and cultural models are of course to take into account to get an answer.
Another level of analysis in Pejtersen's model identifies activities in the context of specific task situations in which complete task repertoire terms, decision making tasks and actors relevant mental strategies are made explicit.In accordance with level 2 in Mantovani's model these aspects emphasize the actors situational and cognitive processes implied during the pursuing of some specific goals.
At a lower level in Pejtersen's framework there is an ergonomic analysis of user interaction with computer systems which considers human perception-action capabilities, testing if an interface specific features match users sensory-motor characteristics as well as perceptual and cognitive resources.This level can be mapped to level 3 in Mantovani's model as it is focused on actors direct interaction with cognitive artifacts through the affordances [15] provided by them.
The comparison of the two models briefly presented (for a more complete description please refer to the bibliography given) aims at demonstrating how research in social and cognitive ergonomics has developed some useful and coherent guidelines to the design of complex information technology systems.We also think that to properly design a system like a multimedia application an analytical path should be followed.The most obvious and appropriate way of approaching the design enterprise is, as we have exposed above, that of starting from a more general analysis of the environmental context (actors and different resources available) in which the system will be used, taking into account the several social factors involved.Then the analysis should focus on more specific aspects of the system design in which actors cognitive, perceptual resources and preferences become relevant and the execution of their retrieval tasks can be decomposed at a more detailed level.
Nervertheless it is hardly surprising that this is not the usually adopted method of performing multimedia systems design since several factors prevent its systematic and widespread application.One reason could be the low scientific status in which the social sciences are at present, as reminded recently in [16].The lack of methodological and easy to apply instruments to perform comprehensively the design task has been another cause of the more pervasive influence of the 'technological viewpoint' during systems development.This fact also explains why the problem of systems evaluation has become so important during recent years to assure a system usability and its acceptance by users.
The evaluation of user-system interaction usually follows the opposite path compared to system design; that is, it follows an empirical approach starting from the analysis of more detailed and low level features affecting interaction to more situational and tasks specific considerations about user performance with information technology applications.
In the following we provide a scheme, inspired by the framework presented here, which includes some important social and cognive factors to consider in designing multimedia systems with particular reference to interface design.We think it could help designer to approach multimedia system design from a wider psychologically informed perspective.

Social and Cognitive Factors in Designing Multimedia Systems Interfaces
In this section we derive from the framework presented before a taxonomy of social and cognitive factors affecting the design of multimedia systems, to show what kind of questions a designer come to face during the design process and how her/his choices can influence interface design.We will also add some comments and indications derived from research on multimedia systems use which have centred on considering users specific way of accessing new data types, such as those conveying visual information.

Goals
Taking a social perspective on design a first factor to be considered is the definition of a multimedia system intended goals.Designers should take seriously into consideration the different cultural models present in a context (their history and implications) when they design multimedia systems aiming at, for example, satisfying educational, entertainments needs or at supporting working activities.From an interface design point of view this means that the choice of the different functionalities to provide can be directly constrained, for instance, by the learning theory and principles adopted (in the case of an educational system), by fashion/market imposed requests (in case of an entertainment system) or by organisational theories and principles (for a work activities support system).
For the case of educational systems the principles derived from a constructivist view of learning give a strong importance to the possibility for the user to not only retrieve multimedia objects (images, sounds and texts) but also to arrange and manipulate them for the purpose of enhancing her/his perceptual and conceptual processes leading to a deeper understanding of the semantic structure of the knowledge domain to be learned.An example of a multimedia system developed to aid students to approach the study of Art History through a direct interaction with images of paintings and through an activity of manipulation of their different elements is presented in Art Explorer description [17].
For what concernes organisational theories we can see how they are responsible for the structure given to working activities.More traditional or tayloristic theories could contemplate that the most appropriate way of supporting an employer activity is by enabling her/his individualized access to a multimedia information system constraining it just to searching and retrieval processes.A different view and consideration of the relevant factors raising the quality and quantity of work production could conceive multimedia systems as means for supporting distributed activities within an organization, the retrieval and sharing of information during work performances and the storing of some specific data elaborated during collaborative processes.This latter philosophy of multimedia interface and system design is represented and discussed for example in [18].

Actors
Another important factor to be considered is the analysis of the characteristics of actors involved in system use.They can belong to different professional groups or communities which have their specific information needs and work modalities; they can be differently committed to the multimedia system even beyond IR needs, for example for maintenance and administration purposes.An understanding of these differences should help to decide for the design of tailorable user interfaces, guiding their development process; in other cases it should promote the adoption of more simple and widespread web-based interfaces or make designers realize the need for collaborative interfaces.Research on multimedia systems use is showing how users preferred ways of accessing new data types (mainly visual information) is by exploring through a browsing activity the collection of documents or by the input of queries that contain multimedia objects.This issues address some important advices for systems designers as they encourage to experiment the adoption of new interface capabilities enabling a better interactivity and cooperation between users and system.For example in [6] it is reported that some problems in multimedia transaction management could be overcome by the implementation of Computer Supported Cooperative Work (CSCW) authoring environments.

Norms and Laws
The social context in which the multimedia system will be introduced must be analysed also as constrained by the norms and laws regulating collective life.In this sense some aspects, relevant to DL development in particular, can emerge as the problem of considering privacy issues and copyright regulation.
For example the recent growing debate on the need of establishing some international agreements on how to deal with authors rights in the digitalized trasmission of multimedia data and products has also shown how the technological development poses new questions that society often is unprepared to answer.Solutions are then found through a negotiation between the different interests and points of view of the institutional parties involved with the problem.For copyright regulation some results have been obtained through the establishment of International Treaties such as the Bern Convention, to which every State legislation can make reference to in dealing with multimedia data copyright.
Privacy issues are especially considered with a specific focus on the treatment of personal data (for example those collected by the system during users past interactions).Many internet software tools and privacy enhancing technologies are at present studied in the international landscape, even if no standardization on this issue has been established till now (for a classification of tools on this subject see [19]).
There is also the need of providing security interface functionalities which can manage multimedia data accessing and retrieval conforming to security norms.From a database perspective in [6] this problem is addressed proposing, for what concernes videos, to implement a multilevel security model using object oriented composition of the universe of multimedia objects.

Integration
Another social factor to be considered is the importance of a multimedia system integration with other systems concerning different collections of documents.This factor is very relevant in domains like medicine or the arts and humanities where the need of finding methods to produce integrated and sharable bodies of knowledge (through the development of widely accepted standards for data representation and cataloguing) is particularly expressed by the DL community.We think that a solution to this problem could derive not only from the development of new and advanced techniques for multimedia data representation but also from the choice of exploiting interfaces and multimedia systems functionalities, enabling a greater and more efficient level of communication and cooperation between actors involved in the process of creating, accessing and updating specific knowledge domains, see [18] for some examples of this kind of applications.A distributed effort based on the negotiation of multimedia data meanings and interpretations could be enhanced by the design of CSCW specific interfaces for collection development and maintenance.The importance of collaboration between users and of the sharing of information (both the search product and the search process) when browsing online information stores is also stressed in [20] where the authors provide also a survey of different systems offering varied approaches to supporting collaboration.
Our point is that the taxonomy of social factors described above (which is briefly summarized in Table 1) can help designers to analyse and understand more deeply the social context of use of multimedia systems expliciting the different assumptions underlining the decisions and choices to be taken.

Repertoire of Tasks
Turning now to consider a classification of cognitive factors that influence multimedia systems design we start by indicating the analysis of cognitive processes involved in the repertoire of tasks the system supports.
A relevant point to consider, first of all, is how the possibility of having access to multimedia data can improve, increase and change the repertoire of tasks a user can perform to satisfy her/his IR needs.From this it follows that some decisions should be taken about which tasks have a priority and should be supported by a specific application.At the interface level we can realize how its layout is structured as to provide representation tools or maps to help users easily access the different type of data formats the system supports.For the case of visual data, interfaces providing bidimensional maps, representing the distribution of images in the database to help image searching and selection, have recently appeared.In [7] it is suggested that a query environment for a visual information management system needs to be able to handle a plurality of possible interpretations of data, embedding both visualization and visual manipulation of data.Some preliminary studies on visual information seeking enabling users to explore large information spaces through dynamic queries, manipulating data and query parameters using graphical objects, are reported in [21].New ways of extending and enriching multimedia data presentation and manipulation at the interface level are currently suggested by many research groups but it is also recognized that they need to be experimented further.

Decision Making
Other interesting processes to anticipate during design are those concerned with decision making aspects in system use.Some methods should be applied to quantify the cognitive effort required for efficient decision making in multimedia IR.This, in turn, should have important influences on deciding about the quantity and quality of information to provide to users as queries results or during navigation.For example it has been observed that in VIR users' judgment of goodness about the system is based on how much of the retrieved data is good rather than on how much of the relevant data is retrieved [22].Users are also particularly sensitive to the ranked order of relevant vs irrelevant objects retrieved [4] and should be allowed more interactivity in reformulating and refining their queries during the interaction process.The need for the development of intuitive interactive tools to quickly access and select relevant multimedia objects is strongly felt also by designers of video database management systems, since videos retrieval can become very complicated due to the huge volume of data and to the fact that their delivery across networks is subjected to bandwidth constraints [23].

Social Factors
Relevant

Mental Strategies
Another factor to study concerns the most commonly used mental strategies users employ to carry out multimedia IR tasks.Interface could be tailored to support a number of strategies (for example analytical search strategy, search by similarity etc.) or designed to enhance interactivity during users searching/browsing multimedia documents through relevance feedback also applied to visual thought supporting functions.

Perception-Action Capabilities
At a more detailed level of analysis designers should consider a configuration of factors affecting users data perception (provided through a visual, textual or auditory modality) and their preferred forms of physical interaction with the system.In interface design this concernes the choices about dimensions, colors, type of letters, icons and other symbols displayed during system use, as well as the design of action buttons and other audio-visual devices prividing the right affordances to users multimedia data elaboration (in Table 2 cognitive factors are resumed).
It is our opinion that following this kind of analysis scheme during multimedia systems design can prove fruitful in eliciting and studying thoroughly both the social and cognitive aspects which have deep implications in determining or preventing a system success with its future users.

Choices Influencing Interface Design
Processes involved in the repertoire of tasks the system supports Processes involved in decision making during system use Mental strategies adopted for tasks execution

Trends in Multimedia Systems Evaluation
The field of multimedia systems design and use is so in a ferment today that we can observe the delivering of new prototypes and systems at an ever increasing rate.Since usability of complex systems is never a 'to be taken for granted' issue we suppose that even multimedia systems evaluation will continue to reserve designers attention in the next years.The process of system evaluation usually starts with the definition of some narrow boundaries for experiments, focusing on detailed analysis of user-system interactions and it takes into consideration more often the analysis of cognitive factors.The tradition of studies on textual IR systems evaluation in the past years has elaborated a well defined set of methods and tools to support system evaluators.A considerable amount of data and experience exists on how to design empirical studies to verify a system usability and adequacy to users needs.The user-centred evaluation methods and tools most commonly used are experiments and observation carried out during laboratory tests, observation in the field performed using on-line and off-line questionnaires, interviews and the analysis of log data.Sometimes designers collect also videorecordings and verbal protocols elicited from users during systems use.All these methods and tools should also be applied to multimedia systems evaluation extending our comprehension about human ability to process, interpret and use multimedia materials.
The outcome of evaluation studies in a cognitive framework could, for instance, suggest designers the opportunity of changing some superficial interface layout features, that is the perceptual/physical affordances the interface provides, or suggest some changing at the level of user system communication, for example in the quantity and quality of multimedia data provided.The results of this type of evaluation studies could also guide system developers to redesign interface so as to provide different or more advanced functionalities supporting users during navigation and searching tasks.
Due to the early stage of development of multimedia applications an extensive user-centred evaluation of their use has not been carried out, yet (at least to our knowledge).The evaluation results presently found in the literature are concerned mainly with precision/recall measures testing and comparing different algorithms and systems retrieval performances.Some informal observations have been collected especially about users interactions with IR systems providing textual and visual types of data.
For the next years we stress the need, within the information science research community, to devote more attention to social factors analysis in systems evaluation, applying a set of instruments from the social sciences to the study of users behaviour in multimedia query environments.
We just list here (see also Table 3) some methods and tools that should be indicated for this kind of studies: starting from ethnographic methodology, we also include survey methods to carry out in naturalistic settings based on interviews, for example with panel or focus groups, representative of users population.We would stress the usefulness of comparing data and results coming from different multimedia systems evaluation to gain insight about users specific needs in interacting with large and distributed collections of documents.Some possible outcomes of this studies could evaluate the importance of improving interface functionalities for communication and data sharing between users.
The consideration of social factors in systems use is actually leading to the enhancement of social and cognitive processes like sense-making and negotiation when multimedia information is accessed and exchanged.In interface design or redesign this means to move from a view of multimedia data as resources to be sought and retrieved to resources that can also be produced and shared among users.

Type of Factors Methods and Tools for Evaluation
Proposed  We think that the framework provided together with the description of a taxonomy of social and cognitive factors affecting interface design can help to analyse the current state of development of multimedia systems and of DL environments in particular, from a user-oriented perspective.
Since the purpose of a DL is to improve the exploitation of the globally networked information universe with a clear focus on the specific information needs of an individual user and his task at hand [24], it is interesting to reflect upon the usefulness of the framework described in helping to reach this purpose.
Our opinion is that the strength of such an enlarged view of actors behaviour in complex systems, like DL environments, consists in giving more emphasis on the social nature of information production, manipulation, retrieval and sharing.A recent trend in DL research field is towards the construction of 'information artifacts' [25], that is, information that can be attached to simpler multimedia objects (like texts, images and sounds) to enhance their representation and retrieval through a network.In the same direction goes the effort of knowledge representation studies which aim at the development of metadata structures representing formalized and explicitly designed conceptualizations of domain contents.For complex and ill-defined domains, such as the social sciences and humanities, the development of specific ontologies could provide useful representation tools to guide non-expert users in accessing large data repositories.Within the emergent field of knowledge sharing in [26] it has been solicited the cooperative and distributed development of formal ontologies as a way of enforcing clarity in the definition of a domain core concepts, among the experts of a specific community.Users of the ontology should also be given the possibility of extending it for their own purposes monotonically, that is, without requiring changes to the base ontology.
We stress here the importance that metadata structures can be directly accessed by users.User interfaces should adopt graphical representations of metadata to assist users in formulating queries and locating data, making easier to understand a domain of data and to correctly manipulate it [27].
Of course a question to be further investigated in the next years is if new collaborative knowledge structuring environments will prove effective and useful to overcome current DL problems in representing and integrating multimedia collections of documents, making easier their retrieval and use.
The usefulness of the framework presented here consists both on the possibility of analysing and considering different kind of data collected during system design and evaluation and on the possibility of deriving from it more specific theories and models about user-system interaction that could be also falsified from the results obtained during empirical studies.
In conclusion we can say that even if reseach on information technology systems design has made some progress in the last decades and many theoretical and methodological data have been accumulated, a system advance and evolution is still linked to an effortful and time consuming process of design-evaluation-redesign cycles.Sometimes the pressure imposed on designers work activities, in terms of time and resources available before products release, cannot provide the right conditions to carry out the different phases of the iterative design, keeping a user oriented framework in mind.
Nevertheless we think also that future research will show if the challenges offered by multimedia systems design and the problems at present faced by the DL field will take advantage of a crossing of boundaries between disciplines and a more evident sharing of competences within designers teamworks.