Heuristics for Course Workspace Design and Evaluation

Course workspace represents the interface between the teacher, the course content, and the student. Both student engagement and student learning are influenced by the course workspace in a similar way as the end-user experience is influenced by system interface. However, course workspaces are typically designed by teachers without a specific input from students, which contrasts the practice in end-user computing and instructional design where users are or should be involved in the early stages of interface design and evaluation. In this paper, we develop a heuristics framework for being applied in a participatory manner involving the student perspective in the evaluation of course workspaces. The framework has been validated on three Moodle course workspaces. The results showed that the framework and the participatory approach provided valuable insights into student experience with the course workspace, while keeping the evaluation effort manageable in terms of data analysis and interpretation. We believe that the heuristics framework and participatory approach could be valuable for teachers, academics, and practitioners that use e-learning platforms for designing course workspaces. The paper provides also examples of improvement areas that have been identified in the evaluation and highlights new research directions in this area.


INTRODUCTION
In the era of online education, course digitalization is commonplace. One element of the course digitalization is the course workspace which represents the interface between the teacher and the course content, the teacher and the student, and the student and the course content. Both student engagement and student learning are influenced by the course workspace in a similar way as the enduser experience is influenced by system interface (see e.g., Meiselwitz & Sadera, 2008;Monari, 2005). However, course workspaces are typically designed by teachers without a specific input from students, which contrasts the practice in end-user computing or instructional design where users are or should be involved in the early stages of system and interface design and evaluation (Preece et al., 2015).
To help teachers create online course environments, there exist numerous electronic platforms, such as Moodleone of the most popular systems of this kind (Moodle, 2017). According to Moodle developers (Moodle, 2018), the e-learning platform is founded on the pedagogical principles of social constructivism (see Palincsar, 1998;Lim & Chai, 2008;Windschitl, 2002), thus by its design the platform provides educators with tools for including the type of resources and activities necessary to build a course that takes into account the student learning and needs. Social constructivism paradigm views learning as being an iterative construction of knowledge and meaning as a result of students and teacher interactions, collaborations, and sharing, and Moodle is built upon this paradigm (Nash & Moore, 2014). However, in most cases in higher education, courses' design and implementation are the responsibility of the responsible teacher (possible involving also other teachers). However, the students are very rarely or never involved in the course or workspace design. Typically, the students' input is utilized in the next course implementation based on the feedback given, but rarely this feedback addresses the student experience with the course workspace.
In this paper, we develop a framework for evaluating course workspaces which takes into account specifically the student learning experience and the usability of the course interface. We validated the framework on three Moodle course workspaces by employing a heuristics-based evaluation approach with the aim to incorporate the student view in improving the workspace. The results showed that our evaluation approach provided valuable insights into student experience with the course workspace, while keeping the evaluation effort manageable in terms of data analysis and interpretation; thus, the framework and approach can be applied as a means for improvement of the workspace in different stages of course development. We believe that the framework and approach could be valuable for teachers, academics, and practitioners that use Moodle or other e-learning platforms. The paper thus provides also examples of improvement areas and highlights new research directions in this area of human-computer interaction.

Electronic learning platforms evaluation
Online learning platforms are built as interfaces for teaching or supporting face to face education. Nowadays when remote education and work became a necessity, these platforms also substitute the traditional face to face teaching modality as well as the traditional social interactions for educational purposes. These platforms are built on top of learning management systems (LMSs). They provide different functionalities and views to their users and provide access every time, everywhere, thus enabling synchronous and/or asynchronous communication. The teachers can store and structure the teaching and learning materials, design and accommodate the learning assignments and the group tasks and interactions, provide feedback and grade the students, store the grades and manage the learning activities. Students can access everything that the teachers provide as learning resources including other students' work in the context of various peer tasks, engage in social interactions with their teachers and peer students, submit their work, view the grades, provide feedback and so on.
To be used effectively by teachers and learners, these systems should possess a series of qualities, and one crucial feature is usability (Rentróia-Bonito et al., 2008). Usability (ISO, 2010) ensures that the learning system can be used by its users (teachers and students) with effectiveness, efficiency and satisfaction towards attaining their teaching and learning goals, respectively. Still, numerous studies evaluating e-learning platforms indicate problems with usability (see e.g., Chua & Dyson, 2004). Furthermore, there is no single framework agreed upon or established to be used for the evaluation of these systems (Ardito et al. 2006;Chua & Dyson, 2004). There are numerous approaches employed such as the use of questionnaires (e.g., Kakasevski et al., 2008;Senol et al., 2014;Zaharias & Poulymanakou, 2009), observation and interviews (e.g., Ardito et al., 2006), and heuristics (e.g., Reeves et al., 2002;Zaharias & Poulymenakou, 2006). There are also diverse frameworks for defining the features to be assessed. These include standard models of usability such as ISO 9241-210 (ISO, 2010) or ISO/IEC 9126 (ISO, 1991) (see e.g., Chua & Dyson, 2004), established heuristics or models such as Nielsen's model (1993) (e.g., Senol et al., 2014, or customized models created for the purpose of the evaluation (e.g., Ardito et al., 2006;Ozkan & Koseler, 2009;Zaharias & Poulymenakou, 2006;Zaharias & Poulymanakou, 2009). Nakamura et al. (2017) reviews existing research that evaluates the usability and user experience of LMSs and provide a summary of methods and constructs employed for evaluation. One notable finding was that while a large majority of studies evaluated the learning factors, these were formulated in different ways across the studies. Learning factors varied from "content relevance" to "interaction between participants", "feedback and orientation", "instructional assessment", "content organization and structure", "motivation", "support for significant learning approach", "media use", and to "collaborative learning", as well as other dimensions (Nakamura et al., 2017). Based on these findings, we formulate the first proposition regarding the evaluation of workspaces: Proposition 1: A course workspace design can be characterized by the following learning dimensions contents' structure, navigation, social interaction and collaboration, teacher feedback and support.
The above learning dimensions are consistent with other findings (e.g., Rentróia-Bonito et al., 2008). Furthermore, across studies, evaluation of usability and user experience in the learning context addresses both technical or interface issues such as navigation, but also pedagogical aspects such as well-structured content and instructions to facilitate learning (e.g., Nokelainen, 2006;Rentróia-Bonito et al., 2008;Squires & Preece, 1996;Zaharias & Koutsabasis, 2012). Usability guidelines regard both the interaction (dialogue) with the system (menu, hyperlinks, structure) and the presentation of the information (clarity, visibility, colours), and both aspects will have implications to both functional and ergonomic acceptance (van Welie et al., 1999). Therefore, we formulate the second proposition.
Proposition 2: Usability of a course workspace design consists of both the technical or interface usability and the pedagogical usability.

Student-or learning-centred evaluation
Student-or learning-centred teaching is the state-ofthe-art pedagogy paradigm in higher education (Postareff & Lindblom-Ylänne, 2008;Wright, 2011). The abstract and complex scientific concepts may be counterintuitive to the students and can be different from the currently held world views (Lehtinen et al., 2020;Posner et al., 1982). Therefore, merely transmitting information does not help in understanding the new concepts, and a participatory and collaborative approach to learning is more suitable. Accordingly, teaching is planned to facilitate students' learning processes, rather than to only transmit information and focus on content (Postareff & Lindblom-Ylänne, 2008). The studentcentred pedagogy is an approach aligned with the constructivist view of teaching and learning (Prawat, 1992). The objective of student-centred teaching is that students learn to form their own knowledge and conceptions about the taught discipline, and eventually change these conceptions when new knowledge, skills, and experience are developed, acquired, and accommodated (Posner et al., 1982;Trigwell et al., 1994). In this conception of teaching, the learning process is effective when the student actively participates in the teaching-learning process (Trigwell et al., 1994). The teacher is responsible of facilitating this learning process by structuring the educational situations and facilitating peer interactions (Trigwell et al., 1994).
In the context of online education, the socio-digital environment design features such as collaboration, topic structure, and feedback play an important role (Hypponen & Linden, 2009;Lim & Chai, 2008;Siklander et al., 2017). Usability aspects such as ease of use, visual appearance, and multimodality influence student participation and interest (Siklander et al., 2017). The time is an important resource used for students in learning (Hyppönen-Linden, 2009), thus the usability of the workspace plays an important role. An intuitive and easy to use course interface design minimizes the cognitive load of navigating the course contents and provides optimal context for learning the subject matter, avoiding situations where the learner's cognitive resources are expended on ancillary tasks of students finding their ways through the interface (Mehlenbacher et al., 2005;Squires & Preece, 1999). Therefore, we formulate the third and fourth propositions.
Proposition 3: The student-focused teaching strategy is implemented in a course interface design by ensuring that the design of social interactions, computer-mediated dialogue including tasks and assignments, and contents' structure facilitate learning and engage students in the teaching-learning process. Good practices of pedagogical usability include providing feedback, facilitating collaboration, providing collaboration opportunities, good contents' structure.

Corollary of P3:
With regards to evaluation, the student-centric evaluation of workspace designs addresses pedagogical usability issues such as social interactions, teacher support and feedback, contents' structure.

Proposition 4:
The student-focused strategy is implemented in a course interface design by ensuring that the design of the computer-mediated dialogue is easy to use and intuitive, and that the visual appearance and modalities are suitably designed to facilitate interest and minimize time and cognitive load.

Corollary of P4:
With regards to evaluation, the student-centric evaluation of workspace designs addresses interface and technical issues such as easy to use dialogue, intuitive visual designs, easy to use navigation.

Usability evaluation
Usability is a complex, multi-dimensional, evolving concept reflecting different facets of how users perceive and experience a product, service, or system (Rajanen & Rajanen, 2020). There are various definitions of usability, and practitioners adopt one or another depending on factors such as culture, background, organizational factors, and system development and usability experience and practice (Rajanen et al., 2017). The existing, alternative definitions (e.g., Folmer & Bosch, 2004;Rajanen et al., 2017;van Welie et al., 1999) highlight different aspects of the interaction with a system such as learnability and freedom from errors (Nielsen 1993;Shneiderman, 1998), social and organizational contextual aspects such as impact on the organization (organizational usability, Hertzum, 2010), the effectiveness, efficiency, and satisfaction of using a system (ISO, 1998), experiential or system-orientated attributes (ISO, 2010;Kujala et al., 2011;McCarthy & Wright, 2004). The international usability standards define usability as being the effectiveness, efficiency, and satisfaction of accomplishing user's tasks in a specific context of use (ISO, 1998). The user satisfaction component of usability evolved into the user experience concept (Bevan 2015) with its own standard definition (ISO, 2010). When applying these abstract definitions in evaluation and design practice, usability attributes and indicators or metrics are crucial (Folmer & Bosch, 2004;Marghescu, 2009). With regards to course workspaces, we formulate Definition 1 based on ISO (1998) and identify the usability attributes accordingly in Proposition 5.
Definition 1: Usability of a course workspace is the extent to which students can use the workspace with effectiveness, efficiency, and satisfaction towards completing the course.

Proposition 5:
The usability attributes of a course workspace are effectiveness, efficiency and satisfaction of using the workspace towards completing the course. Folmer and Bosch (2004), inspired by a model of van Welie et al. (1999) define an intermediary layer between usability definitions and attributes on the one hand, and the design patterns and heuristics on the other hand. This intermediary layer is represented by the usability properties (or usability means) and helps in mapping the low-level usability or user-centred design patterns, principles, and heuristics to higher-level usability attributes or goals (Folmer and Bosch, 2004;van Welie et al., 1999). Examples of usability properties are consistency, feedback, task conformance, user control, guidance, error management (Folmer and Bosch, 2004;van Welie et al., 1999). These are embedded in design heuristics and principles and ensure that usability goals and attributes are achieved. As the usability means are not observable in user testing, they are employed within inspection evaluation methods such as heuristic evaluation for diagnosing and improving usability (van Welie et al., 1999). We formulate the following definitions and the sixth proposition.
Definition 2: Heuristics of a course workspace are low-level propositions that guide the design and evaluation of a course workspace.
Definition 3: Usability means or properties of a course workspace are propositions that link the low-level heuristics to high-level attributes. In other words, the usability properties are means to formulate the low-level heuristics.

Proposition 6:
The usability means of a course workspace are not measurable in user testing, but can be assessed using inspection methods such as heuristic evaluation. Thus, heuristic evaluation of a course workspace is employed to evaluate whether the low-level design solutions are aligned with the higher-level attributes. For this, low-level heuristics are formulated and used in inspection as guidelines.
Among the available methods for evaluation, heuristic evaluation (HE) is particularly suitable for formative assessment, and thus useful for improving the course interface. Empirical studies comparing various evaluation methods provide evidence that HE performs well in terms of effectiveness in identifying usability problems and efficiency in using time and human resources and expertise (Davids et al., 2013;Ssemugabi & De Villiers, 2007). Ssemugabi and De Villiers (2007) compared HE with the survey method in the evaluation of a course website. Their findings showed that the two evaluation approaches were similar in performance, yielding a similar number of problems, though each method produced a considerable amount of unique results. HE found more major problems and used a smaller amount of human resources, though those were members of the academic staff while in the survey the participants were students.
Literature reviews on e-learning evaluations (Nakamura et al., 2017;Sagar & Saha, 2017;Salas et al., 2019) provide comprehensive lists of highlevel attributes and low-level guidelines to be evaluated, as well as insights into the most common methods of evaluation. However, many of these attributes, such as those related to help functions, do not purport to e-learning environments with which students are already familiar. Thus, the multitude of attributes listed in previous e-learning evaluation studies are on one hand overwhelming by their amount and on the other hand just slightly relevant with regard to a course workspace evaluation. Furthermore, the vast amount of studies and frameworks can provide the same type of confusion as the general usability attributes, in that the same concepts are named differently in different studies, while different concepts get the same or similar names (Folmer and Bosch, 2004). We formulate the following research problem.

Research Problem:
There is a need to develop suitable heuristics for course workspaces that map high-level attributes (effectiveness, efficiency, satisfaction) to low-level workspace design solutions (structure of contents, etc.).

Heuristic evaluation
Heuristic evaluation (HE) was developed in the early 1990s as an answer to the calls from the software industry for a discount usability evaluation method, not requiring the resources and infrastructure needed for the traditional laboratory testing and other traditional usability evaluation methods Nielsen, 1993). The process of HE starts with assembling a group of usability or domain experts, who evaluate the design against a list of individual items called heuristics (Nielsen, 1993). The experts first perform the evaluation individually, and then compare and combine their findings into one list of usability problems and proposed solutions for fixing them . HE is a relatively cheap and flexible tool for finding usability issues in all stages of the design process and it can be also conducted by novice evaluators with good results (Nielsen, 1993). Although it is recommended that there are at least five evaluators to uncover 75% of the usability issues, it is possible for also a single evaluator to perform HE, though there will likely be less problems found (Nielsen, 1993). Often these lists of heuristics have been assembled by researchers and practitioners, who try to distil the list into manageable number of best practices to be followed or common pitfalls to be avoided. Today HE is widely used in different domains and there are many lists of heuristics for many different contexts such as software development, web development, game development, as well as online learning.
There have been developed several heuristics that target digital learning platforms, web-based elearning applications and online courses (see e.g., Albion, 1999;Alsumait & Al-Osaimi, 2009;Ardito et al., 2004;Dringus & Cohen, 2005;Georgiakakis et al., 2005;Mehlenbacher et al., 2005;Mtebe & Kissaka, 2015;Nokelainen, 2006;Reeves et al., 2002;Squires & Preece, 1996;Squires & Preece, 1999;Tolhurst, 1992). Most of these heuristics use as basis for evaluation the Nielsen's ten principles (Nielsen, 1994) and adapt or complement them to fit the purpose of the evaluations. As a consequence, the existing heuristics exhibit a high degree of overlap, which means they will uncover similar usability problems as the Nielsen's heuristics (see e.g., Zaharias & Koutsabasis, 2011). On the other hand, the conceptual basis for the new heuristics is not clarified in most cases and the refined lists vary across the different studies as researchers try to overcome the existing gaps.
These shortcomings make it difficult to select one set of heuristics to use for a particular context, and researchers and practitioners prefer to develop new heuristics to fit their purposes. In an effort to advance the development and validation of usability heuristics of e-learning systems, Zaharias and Koutsabasis (2011) compared two representative usability heuristics for e-learning, namely those developed by Mehlenbacher et al. (2005) and Reeves et al. (2002). It was found that the two sets uncover similar number and types of usability problems (Zaharias & Koutsabasis, 2011). However, traditional heuristics such as those related to error prevention seem to be inadequate when employed for evaluating e-learning systems, when these are built upon conventional platforms such as Moodle. In the study by Zaharias and Koutsabasis, no such usability problems were found related to error prevention. Thus, the current heuristic lists, while inclusive in covering a multitude of issues including error prevention and accessibility, may appear overwhelming to evaluators as there are very many principles to take into account. Many issues are relevant to the whole environment, service, or program rather than to the individual course design and interface. For example, the visibility of system status heuristic (Nielsen, 1994) is operationalized in terms of usability property as the means of the system to provide information about what is going on or about the success of an operation such as downloading a file (see Reeves et al., 2002). We assume that these types of heuristics, while relevant for the evaluation of a new system or when comparing two systems or platforms, are not useful to be evaluated when the aim is to improve the usability of a course workspace, especially when the teachers are not able to modify the learning platform behaviour or functionality. We formulate the seventh and eighth propositions.
Proposition 7: Course workspace heuristics should provide means to identify problems with the course workspace design as opposed to the learning management system design.
Proposition 8: Good heuristics are based on previously validated conceptual models that map high-level attributes to low-level design principles.
In order to answer the Research problem that we formulated above, we developed a framework for the usability evaluation of course workspaces, framework built upon the aforementioned definitions and propositions (Figure 1).

Figure 1:
High-level heuristics evaluation framework for course workspace usability (the dashed line separates the general LMS usability from the course workspace usabilitythe focus of this paper) Figure 1 illustrates also the relationships between LMS usability and course workspace usability. The high-level view of the framework depicted in Figure  1 highlights the role of heuristics, the two aspects of usability of a learning workspace (technical and pedagogical), and the role of usability attributes in the framework as lenses.

HEURISTICS EVALUATION FRAMEWORK
In this section, we describe the framework at the structural level. The heuristics framework that we propose for evaluating the course workspace usability is largely inspired by the usability model by Doll and Torkzadeh (1988). Doll and Torkzadeh's model addresses the evaluation of the satisfaction (usability attribute level) with information systems, where the quality of information and presentation is critical for the users. This model consists of five usability dimensions, namely ease of use, content, format, accuracy, and timeliness and has been validated (Doll & Torkzadeh,1989). Thus, based on Proposition 8, we construct our framework on this model as it links high-level usability attributes (effectiveness, efficiency, satisfaction) to low-level propositions such as ("the information is clear").
The framework is structured in a layered hierarchy (Folmer & Bosch, 2004;van Welie et al., 1999), where the top level is the highest-level goal, namely the workspace usability ( Figure 2). The middle layer represents the usability properties (means, criteria, or dimensions of achieving the top-level goal; Folmer & Bosch, 2004;van Welie et al., 1999). The high-level attributes, namely effectiveness, efficiency, satisfaction, are not represented in the figure to keep the model simple as these are incorporated in the definition of usability. The last layer represents the heuristics which are low-level, actionable principles or guidelines that can be evaluated and acted upon to improve the usability of the workspace.
To incorporate both technical and pedagogical usability, we adapted the original dimensions of the Doll and Torkzadeh's model as follows. Ease of use dimension was divided into navigation and social interaction, as these are two factors identified as being related to learning (see Proposition 1). In addition, we included the Learning dimension to incorporate additional pedagogical usability aspects. The framework consists of a model with 7 dimensions as follows: Interaction, Navigation, Format, Content, Accuracy, Timeliness, and Learning. Each of these dimensions is assessed through a number of items (called heuristics); in total 39 heuristics. Each dimension represents a different aspect of the way the course information is presented and interacted with during the course, and ultimately affects the learning experience. Thus, the framework focuses on how information is presented and how convenient it is for the student to navigate through different pages, activities and resources (technical usability). Furthermore, pedagogical usability aspects that address the learning factors are also included throughout the framework to ensure that the content's structure, social interaction and collaboration, teacher feedback and support facilitate and do not hinder the students' learning (Rentróia-Bonito et al., 2008;Squires & Preece, 1996;Zaharias & Koutsabasis, 2012;Zaharias & Poulymenakou, 2009).
The evaluation model is shown in Figure 2, while the heuristics are presented in Appendix. The evaluation dimensions are described below. Examples of aspects to be evaluated are provided and different dimensions are contrasted.

Figure 2: Evaluation framework of the usability of course workspaces
Interaction is about the ease of use of the different tools, tasks and activities of the course that involve social interaction (Doll & Torkzadeh, 1988). For example, here it is evaluated how easy it is to perform a task related to the course such as, submit files for evaluation or feedback, answer questions, or interact with the others. It is not about the level of a difficulty of a learning task, but about the ease of use of the interface or dialogue (Karat et al., 1992): how convenient it is for the student to provide the information requested, to communicate with others (teacher and students), to respond to a given request when needed. Social interaction is also present in various frameworks (see Koohang & du Plessis, 2004;Mehlenbacher et al., 2005).
Navigation is about moving around the course workspace, searching for information from it and browsing its contents (Sagar & Saha, 2017;Zaharias & Poulymenakou, 2006). This is different from Interaction in that here the evaluation is focused on the visibility and accessibility (see Reeves et al., 2002;Koohang & du Plessis, 2004) of the information available in the course workspace and the controls available. Is the information easy to find or quickly available? or are there a lot of user actions required to get to the needed/relevant information?these are examples of aspects evaluated in this category.
Format is about evaluating the format of the course and the structure of the information presented at the higher level in the course workspace. For example, aspects such as look and feel (Reeves et al., 2002), presentation of information (Koohang & du Plessis, 2004), clarity, structure (Doll & Torkzadeh, 1988), visual design (Zaharias & Poulymenakou, 2006) of the course are evaluated. This is similar with the Navigation criterion in that a good Format would naturally lead to an easy, optimal Navigation, however, in this evaluation the focus is on the visual presentation and the structure of the information (visual layout, Karat et al., 1992), rather than the controls used for Navigation. However, the two evaluations (Navigation and Format) can sometimes lead to similar conclusions, but Format should be more informative on the learning effort and experience (presentation), while Navigation should inform more about the ease of use and the effort spent on workspace operation (users' tasks to operate the workspace).
Content is about evaluating the content provided in the course workspace (Doll & Torkzadeh, 1988). For example, students should not need multiple applications to view the content (minimize the use and effects of modes, Karat et al., 1992), and the information should be complete and well-formatted (Doll & Torkzadeh, 1988) and presented in a consistent way (Karat et al., 1992).
Accuracy is about how accurate the workspace is in that the interface is free of errors that would make a student feel frustration, confusion, and disengagement (Doll & Torkzadeh, 1988). For example, when clicking an option or a link, one would get what was expected. Accuracy is also about tracking if there are any errors in the content, for example when the information is not updated, or information about tasks is not clear or accurate.
Timeliness is about evaluating time-related aspects (Doll & Torkzadeh, 1988) of the course such as schedule, progression, and getting feedback.
Learning is about evaluating the aspects in the workspace that are related to directly supporting the student learning (Zaharias & Poulymenakou, 2006), besides the ones already covered in the previous categories. For example, the presentation of the learning objectives, the structure and variety of the learning tasks, strategies and materials (see also Squires & Preece, 1999), the appropriateness of the scheduling of the tasks and activities (Zaharias & Poulymenakou, 2006). If Timeliness evaluates whether schedules are provided and progression is enabled, Learning evaluates whether the schedules are properly defined from the student perspective. Social dynamics (Mehlenbacher et al., 2005) play an important role in constructivist learning, thus they are also included in this category.

VALIDATING THE FRAMEWORK
To validate the framework, we applied it to three course workspaces in order to identify usability issues that can be fixed by the teachers or course designers. Heuristics validation studies adopted this method most often (Hermawati & Lawson, 2016).
For validation, we have employed a participatory approach where the evaluation was carried out by a MSc student specialized in usability and user experience design and evaluation. This evaluator profile was selected according to the goal of evaluation, to incorporate both the student view (user task domain expertise) and usability expertise (design domain expertise) (Paz et al., 2018). The evaluation manager represented the pedagogical expertise. Heuristic evaluation using one evaluator with domain experience is an accepted and viable method in usability field (c.f. Hertzum & Jacobsen, 2001) and it has been successfully used in usability studies on distant learning (Erenler, 2018) and social networks (Toribio-Guzmán et al., 2016), among others. The review by Hermawati and Lawson (2016) reports on three studies that use one evaluator for the validation of the heuristics (Carvalho et al., 2009;Jiménez et al., 2013;Salvador & Assi-Moura, 2010) in two different domains health information systems and grid applications. Furthermore, Paz et al. (2018) reports on ten studies using one or two evaluators. The procedure employed in this study ensured that the evaluation was thorough thus finding the most relevant usability issues for students.
The evaluator was representative for the target user group; he had experience with Moodle from other courses, but it was first time when he was exposed to the three course workspaces to be evaluated. The evaluator had participated in one of the three courses in the past but using a different LMS, thus he was familiar with its contents to a certain extent, but he was not familiar with its current implementation and workspace in Moodle. Thus, the context of evaluation resembled a real situation where a student takes the first contact with a course workspace and implementation. Furthermore, the evaluator had knowledge and skills of usability, user experience, and interaction design acquired from his studies. This background was ideal for this type of student-centred evaluation and the existing literature indicates that students are suitable for heuristic evaluation of learning environments (Albion, 1999;Quinn, 1996). Moreover, prior to evaluation, the courses' responsible acted as the evaluation manager and supervisor. The evaluator became familiar with the chosen heuristics as these were designed, collected, assembled, and discussed in a collaborative manner together with the manager of the evaluation. Post-factum, a senior usability researcher with a long-term experience and expertise in heuristic evaluations reviewed both the heuristics and the evaluation results.
The validation process followed the protocol outlined by Reeves et al. (2002) and has been adjusted to fit the current evaluation context and needs. Furthermore, the validation protocol followed the steps outlined by Paz et al. (2018): planning, training, evaluation, discussion, and reporting. The employed protocol included steps such as planning the evaluation, the evaluator getting familiar with the evaluation task and the target course workspaces, the evaluator participating into the literature review and formulation of the heuristics. In the actual evaluation, the evaluator analysed one workspace at a time in light of the defined heuristics. If necessary, the heuristics were slightly refined and discussed with the evaluation manager. The evaluation was conducted in an independent manner, by reviewing thoroughly every page, activity, link, learning material available in the course workspace according to the defined heuristics. The evaluator rated the heuristic on a scale from 1 (poor) to 5 (excellent), similarly with the approach used in other studies (e.g., Albion 1999). Both good experiences and problems were written down in a table, where in the first column were listed the heuristics, in the second column the scores, and in the third column the suggestions for improvement, the problems or the examples of good practices or experiences. Severity was not assessed, as the evaluation aim was to identify and fix all the problems.
The evaluation results of each course workspace were discussed with the evaluation manager. The discussion included also suggestions to improve the workspace or solve the problems. A report of each evaluation was compiled in a table with the heuristics and the evaluation results found. The reports included the suggestions for improvements.

VALIDATION RESULTS AND RECOMMENDATIONS
The results of the validation are presented in this section by the usability dimensions, namely, interaction, format, navigation, content, learning, accuracy, and timeliness. The detailed heuristics used in evaluation are presented in Appendix.
The evaluation of the Interaction addressed aspects such as submitting assignments (I1), interaction with teachers (I2), interaction with peer students (I3), students giving course feedback (I4, I5, see Appendix). The focus was on evaluating the ease of use of the workspace regarding the abovementioned aspects, or whether it was possible at all for the workspace to afford those tasks. Generally, all three workspaces seemed well designed with respect to Interaction heuristics. For usability to be optimal, it is important to provide a logical design of the interactions and tasks. For example, the return boxes should be logically placed so that they are easy to access by the students (I1), and the feedback forms to be easier to access and feedback to be encouraged throughout the course (I4 and I5). Furthermore, the face to face activities need to be translated into similar online activities that are easy and meaningful to do.
The evaluation of the Format addressed the following aspects: course structure (F1), course format (F2), visual elements in the course workspace (F3), consistency (F4), and clarity and meaningfulness of the course/workspace structure (F5). There were a few usability problems identified that are worth noting. Long pages with a lot of information could be divided into shorter pages, their structure be improved for example by using weeks to divide the content, and redundant information removed (issues with F1). On the other hand, relevant information for the students regarding the course structure can be directly incorporated in the workspace, rather than on a separate document or be made available in both ways to accommodate different students' needs (F2). Consistent visual patterns in presentation seem to affect the user perception of workspace usability, thus elements such as headings and font formatting should be used consistently throughout the course workspace (F3). Consistency in terms of structure and contents are also important for students; thus, if one page in the workspace differs significantly from the others, the usability can be perceived as low (F4). In cases where the course contents and learning objectives dictate the structure, then the workspace designer should ensure there is appropriate guidance and context provided so that the students are not confused or overwhelmed. However, whenever possible the consistency principle should be applied. The consistency of visual elements seems to affect also the meaningfulness, usefulness and clarity of the workspace (F5).
The evaluation of the Navigation addressed the following aspects: moving around the course workspace (N1), finding information in the workspace (N2), augmenting the workspace with hyperlinks and guiding elements (N3), and remembering things in the workspace (N4). There were pointed out usability problems regarding the navigation through a particularly long page in one of the workspaces (N1, N2). Long pages that are not structured meaningfully and demand students to scroll large portions of text seem to lower the navigation experience and increase the time to find the target information, despite its "ease of use". The use of hyperlinks to connect different pages and activities contributes to usability (N3). Page headers are important for students to navigate through the various pages and activities, and they are part of the guiding elements (N4, N5).
The evaluation of the Content addressed the: completeness of information (C1), formatting of the content (i.e., lectures, downloadable documents; C2), ease of access (C3), video services (C4), and consistency (C6). For an optimal usability, it is important that all content relevant to the students be available directly in the course workspace rather than through links to external pages or services (C1). Thus, embedded videos are preferred to linked videos; however, for having more control over the videos, some teachers prefer to have the videos linked. Usability is also influenced by consistency of lecture materials (C2), though in the case of guest lectures the originality of the layout can be beneficial to students' engagement. Easy access to downloadable resources such as .pdf files can be implemented as descriptive hyperlinks that can be open in a browser for quick scanning, rather than using file uploads that require the students to first download the file and then scan them (C3). Suitable file formats also influence the usability. for reading assignments, the pdf format is preferred, but for writing assignments where a template is required to be followed, this should be provided in an editable format (C4). Formatting of the contents, including the way the video materials are aligned in the page, can bring confusion to students that lead to additional cognitive tasks requiring allocating resources for interpretation and attention. Thus, ensuring the presentation of the contents is consistent within a page as well as between pages and activities will increase the usability of the workspace (C5).
The evaluation of the Learning addressed the following aspects: presentation of the learning objectives (L1), sequence of the learning content (L2), guidance and support (L3), hierarchical organization that facilitates learning (L4), visual elements that enhance learning (L5), social interaction implemented in tasks and activities (L6), variety, richness and recency of resources (L7-L9). All workspaces provided a list of learning objectives, but to enhance engagement, learning objectives should be more visible and given more attention from teachers, for example, be formatted and positioned as to capture students' interest (L1). The visual means were limited in the workspace according to the evaluation (L5). In all three courses, the lectures provided various types of graphics and pictures to enhance student engagement and understanding, but more visual means are demanded in the workspace. For example, graphics and charts depicting the lectures and content, the different modalities of completing the course, animated videos or games illustrating concepts or approaches, or interactive tools that engage the students in exploration could be implemented.
Social learning occurs when peers and teachers interact and this interaction can have many forms, including peer-reviews, feedback sessions or filling a form. A real dialogic approach would be optimal for social learning, but this is not always possible, for example when students prefer to do the course independently. From the student's perspective, this independent mode that incorporates only peerreviews may lack social interaction (L6), so more active ways of social interaction can be facilitated whenever possible also for the independent study mode. Question and answer activities are good at enhancing student experience with the course and learning through social interaction (L6).
Variation in resources, exercises, and a flexible approach where students can choose among different tasks and materials increase student's learning experience (L7, L8). Providing different media for lectures (live, video, and text or slides) increases the student learning experience (L7) and accommodates different learning styles and modes (participatory or independent study). However, care should be taken when providing too many resources for similar contents (L8), as students may feel overwhelmed with the information and spend additional time scanning through the resources. Finally, teachers must ensure the learning resources are up to date to meet the student's expectations (L9).
The evaluation of the Accuracy addressed the following aspects: precise and consistent information (A1), precise formulation of the activities (A2), accuracy of hyperlinks or of clicking options (A3), freedom from errors (A4). The accuracy of the workspaces was relatively high; the only problems found were related to some links that did not work in the test environmentthe workspaces have been duplicated for the test, and some links were directed to pages that were not accessible to the evaluator.
The evaluation of the Timeliness addressed the following aspects: evenly scheduling of the activities (T1), timely feedback (T2), knowing what are the next tasks (T3), following own progression (T4), and updated information (T5). One issue raised by the evaluator was the time of the deadline, and accordingly deadlines set at evening are better than in the morning (T1). Feedback by the teacher was not possible to be evaluated in a test environment, but the evaluator suggested that some information regarding the schedule of the grading could be provided (T2). There was suggested that the contents and tasks in one course to be structured by weeks, so that students would know what tasks they are supposed to do next (T3). Though, a more clean and clear structure would help (see Format and Navigation), another design strategy is to use the calendar option in Moodle that points out the coming activities. Furthermore, for students is important to see how much of the required tasks they have completed. This information can be provided by enabling the Activity Completion feature for the return boxes (T4). The information about schedules and time should be up to date already when the course starts and progresses (T5).

DISCUSSION
The current study presented a framework for evaluating course workspaces from the student perspective. The framework was built based on literature review and definitions and propositions derived from it. Furthermore, the framework built upon the conceptual model of usability described and validated by Doll and Torkzadeh's (1988). This model of user satisfaction with computing systems survived the test of time and proved to be useful for evaluating information-rich interfaces. We have incorporated relevant heuristics regarding educational systems and applications. Many of these were adapted from the Nielsen's heuristics Nielsen, 1993) for software evaluation. However, we selected and adapted those heuristics that fit our evaluation goals. The framework incorporates pedagogical usability as well as the technical usability properties. We validated the framework by applying it to three workspaces using a participatory evaluation approach guided by the protocol of Reeves et al. (2002) and summarized into the following stages: planning, training, evaluation, discussion and reporting (Paz et al., 2018). 69 The empirical results showed that heuristic evaluation by student provides valuable insights into student experience with the course workspace, while keeping the evaluation effort manageable in terms of data analysis and interpretation. The evaluation results were also interpreted beyond the reported rating scores and suggestions. Thus, ideas for improvement of the workspaces were provided in the results sections. The recommendations can act as checklists for teachers and course designers in future implementations and new courses.

Implications for course design practice
Though there exist numerous other frameworks and heuristics for evaluating learning environments, many of them include aspects that are out of the teachers' control such as help and errors prevention and recovery, visibility of system status (e.g., Reeves et al., 2002). Compared to other frameworks and heuristics for evaluating e-learning contexts and applications (e.g., Mehlenbacher et al., 2005), the proposed framework includes only aspects that the teacher or course designer can control, thus, the results provide recommendations that teachers can apply. The framework can also be used in introspection by teachers to design and evaluate their own course.
The participatory approach of involving a student in workspace evaluation proved to be very useful and should be considered by course designers especially when the course is fully digitalized, is a new implementation or has a complex structure. A student specialized in usability was in a double role, as evaluator and representative user as recommended in the literature (e.g., Paz et al., 2018;Sivaji et al., 2011). Depending on the available resources, more students could be involved in the heuristic evaluation with the aim to yield more views on the usability and student experience. This kind of participatory approach (see e.g., Kogi, 2006) would make the students active agents and co-creators of these courses, which would increase their level of engagement (c.f. Naylor et al., 2020).
The framework can also be used during the course. In these "live" evaluations, it is not necessary to apply all heuristics, but to concentrate on those relevant for the teacher at that specific moment. For example, a teacher might at the beginning of the course be interested on the format and navigation heuristics to make sure that complexity of course workspace does not hinder learning, while later on the focus might be on pedagogical aspects such as learning and timeliness. This kind of evaluation feedback would allow teachers to adapt and improve the course workspace as needed and overcome problems before they have an impact on students. Furthermore, the framework and heuristics can be utilized in questionnaire surveys, at the end of the course, where more students can rate the items thus contributing at improving the workspace.

Implications for research and future work
Research in the areas of e-learning and computerassisted learning can benefit from the proposed framework. The framework can be adapted to different typologies of courses: fully digitalized, collaborative learning, face-to-face learning.
The individual heuristics can be compared with other lists of heuristics yielding new insights and systematic comparisons for future research and for evaluation or design practice. An open repository of heuristics that can be interactively explored with data-driven approaches such as text mining and visual analytics could be one of the priorities of elearning heuristics researchers and HCI researchers in general to advance the field through dataset and participatory research contributions (see Wobbrock & Kientz, 2016).
Future research should also address relevant accessibility or universal usability aspects (Shneiderman et al., 2018) such as subtitles in the videos and access to the online learning and video materials for visually impaired. Some heuristics lists do include accessibility items to a certain extent (see Mehlenbacher et al., 2005;Zaharias & Koutsabasis, 2012), however we consider that accessibility should be addressed separately in a dedicated framework that defines the heuristics specifically in relation with specific user needs and use cases (see also Shneiderman et al., 2018).
Hybrid methods employing user tests, textual feedback, and automated evaluation methods (e.g., Adepoju & Shehu, 2014;Sivaji et al., 2013;Ivory & Hearst, 2001) can also be integrated with the proposed framework in future work.

CONCLUSION
Course workspace acts as the interface between the teacher and the course content, the teacher and the student, and the student and the course content. Therefore, it is vital to evaluate the course workspaces with regards to learning experience and usability. We developed and tested a heuristics framework on three Moodle course workspaces using a participatory approach. The results showed that heuristic evaluation by a representative student with usability expertise provides valuable insights into student experience with the course workspace, while keeping the evaluation effort manageable in terms of data analysis and interpretation. The heuristic evaluation could be conducted before, during and/or after the course, and be focused on the particular heuristic criteria based on the course type and the course workspace design.