An Economic Approach to Studying Division of Workload in Collaborative Search Tasks

This paper describes the current status of my doctoral research. In my work I am developing a novel approach to studying division of workload in collaborative tasks, with a particular focus on exploring division of labour in collaborative information seeking. My approach is based on the application of an economic game to the problem of workload division. Using this game as a theoretical and empirical model of labour division, my work aims to explore the way in which collaborators negotiate over their individual and collective workloads. I expect my research to shed new light on how people choose to negotiate and organise their individual workloads during collaborative search. It is hoped that this information could be used to inform the design of division of labour policies in collaborative systems, as well as to reason more generally about how people value workloads over other rewards and payoffs.


Author Profile
I am currently a second year student enrolled in the MPhil/PhD programme in the Department of Computer Science at the University of Bath, United Kingdom. I identify myself as a human-computer interaction (HCI) researcher, having completed an undergraduate degree in applied psychology & computing and a master's degree in HCI.
In this doctoral consortium paper I present an overview of the work I have completed thus far in my PhD. I hope that presenting my work at the consortium will benefit me in a number of ways. First, I would like to receive comments on my work from academics and HCI professionals from institutions outside my own, in the hope that this experience will offer alternate perspectives on my work that I have not yet considered. A second benefit will be in defending my ideas in front of an external audience, an experience that will prove invaluable during my final thesis defence. Finally, I hope that my knowledge of relevant topics from HCI, psychology and computer science will allow me to contribute successfully during the discussion of other students' work at the consortium.

RESEARCH BACKGROUND
Recent research has shown that information seeking often occurs as a collaborative endeavour among multiple searchers with a shared information need. Such collaborative information seeking (hereafter abbreviated to CIS) has been observed across a variety of domains, including design teams (Poltrock et al., 2003), patent engineers (Hansen & Järvelin, 2005), and knowledge workers at Microsoft (Morris, 2008). In response to these findings, a number of supportive technologies have been developed by the research community, including SearchTogether (Morris & Horvitz, 2007), Cerchiamo (Golovchinsky et al., 2008) and Coagmento (Shah et al., 2009). These systems offer specialised support for CIS and also promote mutual awareness of activity between searchers. Yet in addition to providing general support for CIS activities, researchers have consistently argued that a key design goal for CIS systems lies in supporting division of labour (DoL) (Foley & Smeaton, 2010;Golovchinsky et al., 2008;Morris & Horvitz, 2007;Shah et al., 2009).
Several noteworthy attempts have been made to facilitate DoL in CIS. In SearchTogether (Morris & Horvitz, 2007), the system automatically divides search results amongst collaborators, with an emphasis on an even distribution of search results among searchers. In Cerchiamo (Golovchinsky et al., 2008), DoL is mediated by assigning search partners to complementary roles, each of which has its own set of tasks and user interface. Finally, users of Coagmento (Shah et al., 2009) can organise their work as they see fit, regulating their search activities using chat, bookmarks, and other tools offered by the system.
Despite the variety of methods employed to distribute work during CIS, there is currently a lack of understanding with regard to how people respond to different labour divisions during CIS. This problem is often reflected in the design and evaluation of CIS systems. For example, during their evaluation of SearchTogether, Morris & Horvitz, (2007) found that participants rarely used the automatic DoL facility as it implied that one team member had the 'right' to divide work and was therefore the 'boss' of the group. Since DoL is a fundamental aspect of CIS, as well as many other forms of collaboration, there is a need to understand how people divide and manage individual and collective workloads, as well as an opportunity to explore the impact of different DoL policies on the process of doing work. I am interested in contributing to this understanding by exploring DoL in more detail. To this end, I am exploring the use of a novel methodology for studying division of labour. This method is based on the ultimatum game, a technique borrowed from experimental economics.

The Ultimatum Game
The Ultimatum Game (UG) is a two-player bargaining game where one player acts as a proposer and the other becomes the responder. In a standard UG, the object of the game is to divide a known amount of some commodity, typically a sum of money in the region of USD10. First, the proposer is given the opportunity to propose a division of the resource. The responder then has the option of either accepting or rejecting the offer. If the offer is accepted, the resource is divided according to the proposer's offer. However, if the offer is rejected, both players receive nothing. Either of these outcomes ends the game. The standard economic model (SEM) of behaviour is based on the assumption that people are motivated by self-interest. This model predicts that the proposer in an UG should offer the smallest possible amount to the responder, who should accept the offer in favour of receiving nothing (as would happen when rejecting the offer). However, studies consistently observe that offers in the UG typically average between 30-40% of the prize pool, with a 50-50 split the modal outcome in many studies (Camerer & Thaler, 1995;Camerer, 2003). Furthermore, responders usually reject any offer below 20% of the total money available, and tiny offers are almost always rejected (Camerer & Thaler, 1995;Camerer, 2003). In other words, findings contradict most predictions based on theories of selfinterest -actual offers are usually much larger than predicted and not all offers are accepted. The game thus serves as a useful tool for exploring interaction in a situation where two or more people must bargain over a shared resource.

The Division of Labour Ultimatum Game
Taking the structure of the UG as a framework to study dyadic bargaining, I present a division of labour ultimatum game (DLUG), where, rather than bargaining over a commodity both players want to keep (i.e., money), players instead bargain over a resource that each wants to lose or complete (i.e., work). This game is based on a simple twist of the standard UG, and is described as follows.
In the DLUG, two players begin with a task that involves completing a number of work items separately. Both players have identical tasks and must complete them in the same manner in order to fulfil the goals of their assignment. The players are then given the opportunity to collaborate, such that the tasks they complete contribute jointly towards the completion of the assignment. This collaboration must first be achieved through negotiating a division of their workload. As in the UG, one player acts as the proposer, who suggests a division of the resource (in this case, workload). The other becomes the responder, who either accepts or rejects the proposal. In the event of acceptance, players can collaborate and work items contribute jointly to a shared project. However, should the responder reject the allocation, findings cannot be shared and each player must do the task individually. Either of these outcomes ends the game.
Theories of self-interest predict that players in the standard UG should seek to gain as much of the available resource as possible. In the DLUG, however, the opposite is true. Based on the SEM, we might expect that if both players are interested in minimising the amount of work they each have to do to complete the experimental task, they should both be seeking to lower their individual workloads by as much as possible. Thus, with a task that requires 10 work items, a self-interested proposer should delegate the maximum possible amount (i.e., 9) to the responder. A responder faced by such an offer should then accept, as the offer confers a 10% reduction in workload. However, on the basis of findings from the UG literature, we might expect that a responder would reject such an offer due to its unfairness.
To my current knowledge, the method I have described is a novel way of studying division of workload. The question remains as to what behaviour might be like in this game, and to what extent findings might mirror those using the standard UG. The case of collaborative search thus offers an ideal opportunity to explore this method using a test case where understanding DoL is an important issue. Using the DLUG game as an empirical model of workload assignment, I plan to conduct a series of studies exploring DoL in collaborative search. My intention is to explore how negotiations are managed and how allocations feed forward into the actual work process. At the time of writing I have completed an initial exploratory study using the DLUG in a collaborative search task, and the remainder of this paper gives an overview of the method and most interesting findings from the study.

Overview
The purpose of this study was to explore behaviour in a one-shot DLUG while establishing a methodological paradigm for my research. Ten pairs of participants completed the study, each of whom had to find 10 sources that could be used to answer the following question: "To what extent can design be considered a psychological process?". Participants worked in separate offices and communicated anonymously via text chat. After being randomly assigned to the roles of proposer and responder, participants were informed that they had to search the Web in order to form a bibliography containing 10 sources. They were told that they had the opportunity to complete the study in collaboration with another person. If they wished to collaborate and pool their findings, the proposer was asked to make a workload allocation, which the responder then accepted or rejected. Participants then went on to complete the search task, pasting sources into the text chat as the task progressed. Participants were free to leave the study after completing their individual allocations.

Workload Divisions
All of the offers in the study were accepted and there were no rejections. Of these, 8 out of the 10 proposers made a fair split, with 2 offering a 6-4 split. Such a high number of fair offers suggests that there may be a strong equity norm for division of labour in collaborative search. However, it is not yet possible to say that this established with any certainty, and the findings should be interpreted with caution due to the low number of subjects in my study. It is also possible that the equity was the result of strategic behaviour: responses to post-experiment questionnaires suggested that some proposers made fair offers to avoid the threat of rejection.

Work Process
I found that both of the unfair splits were implicitly renegotiated during the actual process of work. In both cases, the participant with less work stuck around to help their partner complete the assignment. I attribute this to a kind of positive reciprocity, where the person with less work helps their partner as a sign of thanks for completing a greater share of the task.

Search Times
I also measured the total time spent searching; from time of first search query to time last result obtained. I found that there was a high degree of similarity for within-pair search times, but times between pairs were dissimilar. For example, pair 6 completed the study quickly, with an average time of just 5 minutes 49 seconds, whereas pair 1 took almost 30 minutes to complete the same assignment. This effect was remarkably consistent among pairs, and occurred in spite of the fact that participants all had the same task and were not given any performance criteria. It is also interesting that at no point was there a pair whose search times were greatly dissimilar (i.e. one participant taking 5 minutes, the other 30). While I am still exploring this effect in more depth, the findings point towards some kind of emergent synchrony effect, in turn speaking to the possibility of teasing out interesting aspects of collaboration with the method I have developed.

FUTURE WORK
By the time of the consortium, I hope to have completed a second study which will build on the findings I have previously described. More specifically, I plan to examine allocation behaviour by introducing a more complete model of the taskin my first study, participants did not know the search topic until after they had divided the work. My second study will provide complete task knowledge up-front, and will explore how participants' appraisal of the search task affects both workload allocations and the process of work. One might expect that participants faced with a particularly aversive task will attempt to make unfair offers -yet if a similarly high number of fair offers were observed with an aversive task, this would provide further evidence for the strong equity norm in labour division.
I also plan to explore the synchronisation effect in more depth by probing its cause and testing whether it is an artefact of the first study. It is hoped that a wider variety of allocations will work towards this goal, as well as continued analysis of individual search activity. In the longer term, I plan to conduct a study involving an iterated negotiation in order to explore how learning and repetition affect allocation behaviour.

EXPECTED CONTRIBUTIONS
It is hoped that my research will contribute towards answering a number of high-level research questions. The first of these questions is: how do people negotiate and subsequently manage their individual workloads during collaborative information search? By using a simple economic game as an empirical model of labour division, it is expected that my thesis will shed light on this question by exploring how initial allocations are managed and how these feed forward into the process of completing work.
Second, is equity a natural preference for dividing work during collaborative search? The model I provide allows for this question to be addressed via explorations of fairness in workload division.
Third, how do different divisions of labour affect the actual search process? In my exploratory study, I saw that unfair allocations were implicitly renegotiated as the collaboration developed. Further studies using the DLUG will contribute to exploring this question, with a focus on how initial workload allocations are renegotiated and managed during protracted CIS activities.
My work is also the first that has explored reduction of workload as a payoff in an economic game-type situation. This attacks an important question: to what extent do people reason about workloads in the same way that they reason about other resources? Testing whether well known experiments still apply when work is substituted for money is a good way to get at this issue.
Finally, it is hoped that the findings of my thesis will lead to an improved understanding of how DoL impacts collaborative search, in turn informing the design of future CIS systems. In the longer term, I am hopeful that my work might kick-start a broader investigation of how people divide work and their preferences for fairness in labour division.

ISSUES TO BE ADDRESSED
There are a number of issues that I still need to address in my research. Of particular importance will be to consider whether participants in my studies derive utility from doing work (i.e. enjoying the task in its own right) or getting work done (i.e. enjoying sense of relief/achievement when completing work). This distinction will affect both offers made and process of work. An additional challenge lies in selecting the correct experimental manipulations for my studies. The DLUG is highly amenable to variation and I will not be able to explore every possible avenue during my PhD. Also, I have been advised to be wary of becoming paradigmlocked; I need to consider alternate approaches after exploring the current problem space, such that my work remains a study of collaborative practice and not an excursion into experimental economics.

ACKNOWLEDGEMENTS
I am grateful to Prof. Stephen Payne for advising me on this work; to Dr Leon Watts and Dr Geoff Duggan for further support and advice; and to Laura Benton, Pawitra Chiravirakul, Daniel Gooch, Simon Jones, and James Maidment for research assistance.