Derivation of knowledge structures for distributed learning objects

Knowledge space theory (Doignon & Falmagne, 1985; Albert & Lukas, 1999; Doignon & Falmagne, 1999) offers a rigorous and efficient formal framework for the construction, validation, and application of e-assessment and e-learning adaptive systems. This theory is at the basis of some existing e-learning and e-assessment adaptive systems in the U. S. and in Europe. Such systems are based on a fixed and local domain of knowledge, where fixed means that the domain does not change in time, and local refers to the fact that the items are stored and available locally. In this paper we present some theoretical notes on the efficient construction and application of knowledge spaces for knowledge domains that are both dynamic and distributed in space. This goes in the direction of an exploitation of new technologies like the GRID for building the next generation of learning environments.


INTRODUCTION
Knowledge space theory (Doignon & Falmagne, 1985;Albert & Lukas, 1999;Doignon & Falmagne, 1999) offers a rigorous and efficient formal framework for the construction, validation, and application of e-assessment and elearning adaptive systems.It is mainly a psychometric theory for the assessment of knowledge; however, as adaptivity and optimal learning paths does matter, its methods and models can be fruitfully exploited for e-learning purposes as well.
According to this theory, a domain of knowledge is a collection Q of items (e.g., learning objects, problems, questions, exercises, examples, etc.) in a given field of knowledge (e.g., mathematics, physics, chemistry, biology, etc.).Then, the knowledge state of a student is the set K of all items in that this student actually masters, and a knowledge structure for is a pair in which is the collection of all knowledge states that can be observed in a certain population of students.If is closed under union (i.e., whenever ) then it is called a knowledge space.Sometimes also closure under intersection holds ( ), in which case, is called a quasi-ordinal knowledge space.

K , K
The above-mentioned theory is at the basis of some existing e-learning and e-assessment adaptive systems in the U.S. and in Europe.Two of them are the ALEKS (Adaptive LEarning with Knowledge Spaces) system developed by the research group of Irvine, CA supervised by Falmagne (http://www.aleks.com),and the RATH (Relational Adaptive Tutoring Hypertext) system of the research group of Graz, Austria (Hockemeyer et al., 1998).
Such systems are based on a fixed and local domain of knowledge Q , where fixed means that Q does not change in time, and local refers to the fact that the items are stored and available locally.The original theory, in fact, provides methods of construction of knowledge structures which are based on these two requirements (Koppen & Doignon, 1990;Dowling, 1993;Koppen, 1993).Later developments of the theory provide some methods of construction for the case of dynamic domains of knowledge (see, e.g., Albert & Kaluscha, 1997;Stefanutti & Koppen, 2003).However, the efficient construction and application of knowledge structures for distributed (thus, non-local) items appears to be still an open question.
New technologies like the GRID provide a promising approach for the construction of and application of e-learning systems (Ritrovato, 2003).These technologies are strongly based on the concept of distributed services.In this paper we present some theoretical notes on the efficient construction and application of knowledge spaces for knowledge domains that are both dynamic and distributed in space.This means that the learning objects may reside on different locations and that every location has access to both local and remote learning objects.Moreover, the set (repository) of learning objects in the various locations is assumed to be dynamic in the sense that learning objects may be continuously added to (resp.removed from) the local repositories.At the basis of our approach there is the concept of a distributed skill map, which is the subject of next section.Stefanutti, L., Albert, D., & Hockemeyer, C. (2005).Derivation of Knowledge Structures for Distributed Learning Objects.In P. Ritrovato, C. Allison, S. A. Cerri, T. Dimitrakos, M. Gaeta & S. Salerno (Eds.),Towards the Learning Grid: Advances in Human Learning Services (Vol. 127 Frontiers in Artificial Intelligence and Applications,.IOS Press.

is a triplet
in which Q is a nonempty set of items (problems, learning objects, exercises, questions, etc.), is a nonempty set of skills, and is a mapping such that Q .The interpretation of A is the following: for any in , q A Q A q is the set of all skills that (according to ) are sufficient to solve item .

A q
A knowledge structure can be easily derived from a skill map .By the conjunctive model of Doignon (Doignon, 1994;Doignon & Falmagne, 1999) In words, every item in q K is solvable by (some subset of) skills in T , and K contains all such items.It turns out that the collection of all subsets fulfilling (1) is a knowledge structure close both under union and under intersection -i.e., a quasi-ordinal knowledge space.
where is a collection of skill maps, and " " (read: "join") is a binary operator such that, given any three skill maps and T is a quasi-ordinal knowledge space immediately follows from the fact that T is, indeed, a skill map.Moreover, since the join-closure of any distributed skill map , is a complete distributed skill map, the knowledge space delineated by is A A : Thus, a distributed skill map can be seen as a distributed representation of a (quasi-ordinal) knowledge space.

CONSTRUCTION AND RETRIEVAL OF DISTRIBUTED SKILL MAPS
Physically, the elements of a distributed skill map , reside in locations that are separate in space, or they can be produced at different time points.The subset of all elements in that are accessible to a given user in a specific location is the distributed skill map accessible to that user.

Locally, a skill map
is represented as a collection of pairs P q, , where q is an item, is a set of skills such that , and is called a local repository of the skill map .To create a local repository one needs to specify the pairs A P q, , each of which is a single item (problem, learning object, etc.) with a set of skills 'attached' to it.The only requirement is that the items in a local repository have to be unique (to assure consistency of the mapping A ), i.e., for all A Q q , P P P q and P q , , Technically, appropriate metadata schemas have to be defined for the representation of the elements of a local repository (i.e., the pairs ).

P q,
The user is allowed to perform three kinds of operations with local and remote repositories: 1. insert/delete/edit items P q, in the local repository; 2. retrieve items from the local repository; 3. retrieve items from a remote repository.
Through these three operations the user derives, in fact, a local skill map and -implicitly -a local knowledge space which can then be used for e-learning and/or assessment purposes.Thus, the advantage of distributed skill maps is that the corresponding knowledge structure can be derived locally for the specific subset of items collected by the user.

ASSESSMENT OF THE KNOWLEDGE STATE OF A STUDENT
Given a set Q of items, the knowledge state of a student is the collection K of all items that this student masters.
Here we face the problem of how to determine this collection for a certain student in an efficient way.Knowledge space theory provides procedures for the efficient assessment of the knowledge state of the student provided that an explicit representation of the knowledge space is available to these procedures (see e.g., Falmagne and Doignon, 1988).
When the items in are distributed in space the derivation of an explicit knowledge space for turns out to be a rather inefficient approach.To see this, imagine that a user collects a set of elements from different (local or remote) repositories creating, this way, a local skill map (see previous section).To use this skill map for assessing a student with the procedures above-mentioned the user needs to derive the explicit knowledge space corresponding to .This turns out to be a time and space consuming task.

A
At least for quasi-ordinal knowledge spaces, an efficient assessment procedure can be derived which does not require the explicit construction of the knowledge space corresponding to a certain skill map .In this section we provide a sketch of this procedure.

A
First, we define a binary relation " " on Q such that, for any two items The interpretation of such a relation is the following: "the fact that a student masters item p can be surmised by the fact that this student masters q " or, equivalently, "the fact that this student does not master can be surmised by the fact that s/he does not master q p ".The introduction of the relation " " has a clear aim: it is a classical result in lattice theory that there exists a 1-1 correspondence between the quasi-ordinal knowledge spaces on a set Q and the surmise relations for Q (Birkhoff, 1967).

A
Then we introduce the following notation: for a subset , we denote with , and with Suppose now that the knowledge state of a student has to be uncovered through the skill map .This occurs in a stepwise assessment procedure where, in each step the student is presented with a new question and a representation of the knowledge state of the student is updated according to Q q the student's answer.Under the assumption that at the outset ( 0 n ) there is no prior knowledge about * K , the assessment algorithm is as follows: The algorithm sketched above terminates in a finite number of steps A Q m and -if no lucky guesses or careless errors occur during the assessment process (see in this connection Falmagne and Doignon (1988)) -the knowledge state of the student is correctly uncovered by the procedure.One note on the efficiency of this procedure is that, especially when

VALIDITY OF DISTRIBUTED SKILL MAPS
A skill map is a (formal) cognitive model which specifies the skills underlying a certain set of items.As a model it is a hypothetical representation of the relation between items and skills and it needs to be validated.A first step of validity is reliability.The different skill maps in a distributed skill map are supposed to be created by different sources (human experts, teachers, educational institutions, etc.).Each source creates an item and provides a skill assignment to that item.When the same item q is provided by two or more different sources, the problem arises of testing the agreement, among these sources.One question here is that, given a skill map and an item q is the set of all skills in that are sufficient to solve item q according to .Thus, two different skill maps and The ratio between the number of such agreements and the theoretical (maximum) number of agreements for all items in Q , and all skills in provides a raw measure of the total agreement between the two skill maps and B (for the calculation of raw agreement indexes and their statistical significance see, e.g., Übersax, 1993).
Finally, note that reliability is a necessary but not sufficient condition of validity.This implies that even if reliable, a skill map should then be tested against empirical data.If the knowledge space corresponding to a given skill map is considered, then the methods and probabilistic models provided by Doignon and Falmagne (1999) can be applied for this purpose.

CONCLUSIONS
Knowledge space theory provides formal models and methods for the efficient assessment of knowledge in a certain domain of knowledge.Existing methods, however, are applicable only in the case of a local domain of knowledge.An approach for the assessment of knowledge through knowledge spaces, which is appropriate for distributed and dynamic learning objects was presented in this paper.The approach focusses on the construction of distributed skill maps (and corresponding knowledge spaces) and on the derivation of new local skill maps from the distributed ones.The concept of a distributed skill map allows to exploit the advantages of applying knowledge spaces in knowledge assessment when the learning objects are located in distributed repositories, rather then local.This would allow the exploitation of new technologies like the GRID for the construction of e-learning and eassessment systems.
An algorithm for the efficient assessment of knowledge through distributed skill maps was sketched in Section 4. The algorithm applies in case of skill maps generating quasi-ordinal knowledge spaces.The explicit derivation of a knowledge space from a (local or distributed) skill map is time-consuming, especially when the domain of knowledge gets large.The advantage of the proposed algorithm is that its application does not require the explicit derivation of the knowledge space corresponding to a given skill map.
In a distributed skill map, new local skill maps can be obtained integrating existing ones.This integration poses the problem of the validity of local skill maps.This problem was discussed in Section 5 where some basic methods were proposed for obtaining statistical indexes of agreement among different skill maps.Finally, we would like to point out that the approach proposed here is restricted to the distributed construction and application of quasi-ordinal knowledge spaces, which are a special class of knowledge spaces, and knowledge structures.In the case of local learning objects, the concepts of skill multimap and skill multi-assignment were introduced to deal with the general case of any knowledge structure (see, e.g., Doignon, 1994;Düntsch & Gediga, 1995).Based on these existing concepts, further work is needed to extend the concept of a distributed skill map to knowledge structures in general.

[
Note that, from a mathematical point of view, a complete distributed skill map is a complete join semi-lattice in which the join operation is " ".]It follows that the knowledge structure delineated by a complete distributed skill map , * having top element T , is T where, for any opinions on the sufficient skills of a given item.A perfect agreement arises when, for two sets X and Y , let denote their symmetric difference, i.e.,