Introduction: Searching for Value in Distributed Problem‐Solving Networks
The application and diffusion of the Internet and World Wide Web has greatly expanded the opportunities for collaborative distributed working and the sharing of information and expertise. For instance, ‘open source’ software developments, where groups collaborate in software production for non‐commercial or proprietary reasons, seem to defy conventional wisdom about the incentive structures required for the production of high‐quality computer software. This has been the basis of a number of open source project successes (e.g. the Linux operating system) as well as failures.2 The collaborative creation of Wikipedia to a level of quality that has been compared with the Encyclopaedia Britannica provides another illustration of the potential of non‐proprietary, open co‐production of a new product.3
A growing number of researchers view these kinds of developments as illustrating the value of tapping into the ‘wisdom of crowds’—the idea that ‘the many are smarter than the few’. This occurs where it is claimed a large number of ‘ordinary’ people can outperform a few experts by sharing information and solving problems in what has been categorized with labels such as ‘peer production’ and ‘co‐creation’.4 This paper critically assesses this thesis, based on a set of case studies designed to range across a spectrum of distributed problem‐solving activities.5 In particular, the case studies focused on trying to identify the locus of any value in these networks and who gains the benefits. Understanding the most appropriate model for the structure and dynamics of these networks was also a key aim, such as the degree to which the metaphor of a ‘crowd’ or a more differentiated networking of individuals is more appropriate.
The paper starts by highlighting the research questions motivating the study and the methodology employed. It then briefly looks at historical precedents to the new socio‐technical organizational forms identified through this research, which we have called Collaborative Network Organizations (CNOs). A typology formulated through the analysis of the case studies forms the backbone of the research. This categorization is defined, followed by an outline of how the cases fit this framework. The findings emphasize the important role played by forms of management and control in gaining value from CNOs. Among the key research conclusions emphasized is that ‘managing the wisdom of networked individuals’ is more significant than the notion of the wisdom of crowds. However, approaches to the management of CNOs depend on the ways in which they are used to reconfigure access to information and people—the type of CNO they define.
The Key Research Questions
The growing importance of collaborative networks has been the focus of a wide range of research in organization, Internet, management and economics studies. Table 1 summarizes how researchers have sought to explain the value and potential of these networks.
Compared to information systems in formal organizations, CNOs can:
|
| Notes: a This is known as ‘the Jury Theorem’. See: M. de Condorcet, Essai sur L'application de L'analyse a la Probabilité des Decisions Rendues a la Pluralité des Voix, l'Imprimerie Royale, Paris, 1994 [1785].b S. E. Page, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies, Princeton University Press, Princeton, NJ, 2007.c C. R. Sunstein, Infotopia: How Many Minds Produce Knowledge, Oxford University Press, New York, 2006.d W. H. Dutton, ‘Through the network of networks—the fifth estate’, Inaugural Lecture, Examination Halls, University of Oxford, Oxford Internet Institute, Oxford, 15 October 2007. Available at: http://ssrn.com/abstract=1134502. |
In order to examine claims such as those in Table 1, and to better understand the use of collaborative networks in a variety of spheres, this study focused on identifying a set of critical issues that set these networks apart from traditional approaches. For example:
- •
What are the sources of value in collaborative networks and who captures the benefits—individuals, firms, or providers of the network platforms?
- •
To what extent does intelligence in collaborative problem‐solving networks lie in the ‘wisdom of crowds’, or the more differentiated contributions of individuals?
- •
What are the main types of network and what are their differences and commonalities in terms of their social, organizational, and technical underpinnings?
- •
How can the performance of networked approaches to co‐creation and co‐production be measured to enable comparisons to be made between each other, and with more traditional approaches for service and product creation or production?
- •
How do these new forms of collaboration shift traditional balances of power, influence, and authority, for example in the suggestion that sharing rather than hoarding information should be a new maxim? Will any such change undermine the competitive position of firms and the individuals within them—or open a promising new approach to solving problems that will enhance the performance of individuals and their groups and organization?
- •
What are the most significant underlying reasons for failures in establishing and maintaining co‐production networks (e.g. some open source software projects)?
Historical Precedents for Collaborative Network Organizations
The emergence of CNOs represents the latest stage in a 40‐year thread of initiatives using computer‐based systems to harness distributed expertise. For example, the development in the 1960s by the RAND Corporation of Delphi techniques in forecasting6 sought to reduce the bias created by influential individuals in the social dynamics of co‐located face‐to‐face groups of experts. However, the perceived value of these approaches was undermined by difficulties in soliciting thoughtful responses from experts, and by many weak applications of the technique.
The potential for computer‐based communication networks to enable the sharing of expertise accelerated the drive towards distributed collaboration in the 1970s, such as with computer conferencing,7 group decision‐support systems and later initiatives around computer‐supported cooperative work. The diffusion of personal computers across organizations also led to the development of ‘groupware’ and other applications to reconnect individuals within and across organizations through networks.8
More recently, the application and diffusion of the Internet has greatly expanded the opportunities for distributed working and the sharing of information and expertise. Confidence in collaborative networking has been growing. For instance, so‐called ‘Web 2.0’ applications have generated a wide range of proposals for employing ‘user‐generated content’ and greater collaboration in a number of sectors, from social networking to corporate communication and scientific research.
The Research Methodology: Exploratory Case Studies
To address the questions outlined here, the OII–MTI study assembled a team to conduct a set of case studies in fields ranging from high‐energy physics and biomedical sciences to IT software and entertainment. These sought to provide empirical evidence to ground debate over the performance of CNOs and the motivations supporting participation in them. The research itself became a distributed problem‐solving group of 15 academics spanning three continents. This included academics and practitioners with expertise in many fields, including communication, computer science, economics, and management.
A key initial effort involved grouping the cases into categories of approaches to distributed problem solving, such as by distinguishing broadcast search from user rating systems. Early on, the cases were selected to enable us to investigate approaches to assessing the performance of existing networks, such as by choosing cases involving different kinds of products. This six‐month exploratory study linked a continuing review of the existing literature to empirical studies of selected cases that employed different forms of distributed problem solving to create different products. The study used a series of collaborative workshops to critically assess the selection and analysis of cases, while the team also devised approaches to evaluate their comparative effectiveness.
Cases were chosen to provide original insights on different types of networks or organizations for distributed problem solving. Another criterion was our ability to gain access to these networks for more in‐depth analysis. We began by identifying projects, such as open source and Wikipedia, that have become identified with peer‐produced, distributed problem solving. We then sought more novel cases that employ different approaches in a variety of areas of application, from scientific collaboration to film production. For example, after scanning the horizon for an overview of distributed co‐creation in global media and entertainment industries, we focused on one case that we could explore in depth.
The scope of the project was widened by incorporating insights from additional cases, not all of which we studied directly. In the area of ‘broadcast search’ networks for instance—typified by the question: ‘Does anybody on here know someone or something?’—we relied on published work regarding a well‐examined case.
We established overviews of key relevant issues tied to the specific area in which each case is embedded (e.g. indicators and determinants of performance outcomes). Different approaches were then adopted to suit different cases. For example, some cases were focused on a single platform, while others were based on a type of platform, such as news aggregation or prediction markets.
Classifying Types of Collaborative Networks
While each case was incorporated for its representation of a unique set of projects, we also sought to use the studies to group the cases within a more general typology.
An Initial Broad Categorization
A broad categorization into two types emerged from our early scanning of projects.9 One type identified projects that linked individuals within an existing community or organization via Internet‐enabled applications that aim to solve particularly complex and novel problems, such as addressing the ‘bugs’ in software. The second type focused on problems pre‐structured by Internet platforms that enable new inter‐organizational networks to generate or mine insights gathered from the interaction of distributed actors, such as between medical specialists.
There are other useful classifications concerning the nature of the problem being addressed. For example, Page10 identifies three types of problem‐solving networks: those focused on ‘information aggregation’; ‘prediction model aggregation’; and ‘problem‐solving’. However, the case studies underscored the degree to which any categorization system based on the function of bringing together a distributed group of people would over‐simplify the goals and objectives of the actors. The case studies reveal multiple goals and objectives behind the often complex ecology of actors shaping their design and use. They encompass the addressing of many different problems, some simple, others complex. This makes it unrealistic to group the cases by any specific category of problem.
In fact, it was clear that the choice of network was not rationally driven by solving a pre‐defined problem. Often, a network becomes a solution space looking for emergent problems to solve. In contrast to rational problem‐solving models, we more often saw network ‘solutions’ looking for problems to solve. This is similar to the ‘garbage can’ model of organizational decision making, in which people in organizations have solutions looking for problems to which they can be applied, such as outsourcing a problem.11
Instead of characterizing the underlying purpose of complex sets of technologies and activities, we found it is more useful to identify the types of ICT networks that underpin the potential of these collaborations. This categorization was complicated by the degree to which each network exhibited multiple and overlapping design features, and that many design options could be applied across a wide array of industries, enabling the tailoring of collaborative networks to meet the needs of specific communities and problems. Nevertheless, a simple typology emerged.
A Classification Framework for Collaborative Network Organizations
The most prominent design feature that emerged from analysis of the project's cases was that each aimed at reconfiguring who communicated what, to whom, and when within the network. The cases demonstrated that these are not ‘crowds’ involved in collaboration, but regulated interactions among networked individuals—regulated in part through the architecture of the network. Moreover, it became clear that in most of these networks it was a small minority of ‘core participants’ who represented a majority of the contributions made within the network.12 It is therefore more reliable to characterize these networks by the activities they support rather than the purposes they serve. That is why this paper refers to them as ‘collaborative network organizations’, instead of the study's original view that they should be conceived of as ‘distributed problem‐solving networks’.
By identifying a CNO's architecture as a primary design feature, the case studies surfaced a technologically and socially relevant typology that is simple but analytically powerful, based on the evolving nomenclature surrounding different generations of Internet Web technologies and applications. Definitions of these terms vary widely, but we found three main characteristics of CNOs' use of Internet technologies to be of most value for our studies:
From these definitions we were able to identify three types of CNO, which focus on supporting collaboration through:
1.0 Sharing: the ability to create linked documents and objects within a distributed network, thereby reconfiguring how and what information is shared with whom. This is exemplified by Tim Berners‐Lee's invention of the Web to share documents at CERN, which has been moved forward by his later articulation of the idea of a ‘semantic Web’ to support more intelligent search, linkage and retrieval of information.15
2.0 Contributing: the ability to employ social networking applications of the Web to facilitate group communication, thereby reshaping who contributes information to the collective group.
3.0 Co‐creating: the ability to collaborate through networks that facilitate cooperative work toward shared goals (e.g. joint writing and editing of Wikipedia), thereby reconfiguring the sequencing, composition, and role of contributors.
Table 2 below, illustrates the key features of different types of collaboration networks. These overlap. For instance, networks enabling user‐generated content also exploit the hypertext linkages so valuable in finding and sharing documents. Likewise, cooperative joint collaboration—enabled by collaboration 3.0—exploits the potential for user‐generated content as well as hypertext links, at the same time as focusing on the collaborative production of documents or other information products. Many Web 1.0 applications are one‐to‐a‐few or one‐to‐many and are oriented towards broadcasting or narrowcasting information, but without incorporating user interaction as a central component of their operation.
| Collaboration | Hypertextual | User‐Generated | Cooperative Work |
|---|---|---|---|
| 3.0 Co‐creating | × | × | × |
| 2.0 Contributing | × | × | |
| 1.0 Sharing | × |
This classification of three types of collaboration networks should not mask the degree to which each type is embedded within a broader array of communication networks and channels. For example, the use of the Internet to support collaboration on a project should recognize that travel for face‐to‐face meetings can also be central to the team's working, with online and offline contributions often combining to forge a community that is in many ways analogous to a social movement.
The Cases Classified by Collaboration Typology
Networks for orchestrating distributed intelligence tend to focus on one of the fundamental strategies for reconfiguring access pinpointed in Table 2. A summary of the cases we studied is provided in Table 3 opposite, grouped according to these classifications.
| Type | Case | Application for which case is an exemplar | Focus of case |
|---|---|---|---|
| 1.0 Sharing | Atlas | Design and management of a large‐scale research project sharing information between many researchers | Web‐based documents supporting collaboration among 1,900 physicists in 37 countries working on a high energy physics experiment |
| Bugzilla | Use of shared, viewable database for coordinating distributed collaboration | Database tracking software defects and managing repairs for Firefox and other Mozilla open source projects | |
| InnoCentive | Broadcast search: networking problem holders and solvers through awards, prizes, and other incentives | Solution ‘seekers’ compete for prizes and generate solutions through broadcast search processes | |
| Neurocommons | Deep search: enables both documents and data to be linked and searched | Access to biomedical information through deep searching and natural language processing of open abstracts and datasets | |
| 2.0 Contributing | Digg and other news platforms | Aggregating news content | News aggregator finds, rates, prioritizes online news |
| Sermo | Sharing insights, information and opinions among experts in a field | Licensed physicians in USA share information and assist each other and sponsoring organizations | |
| Information markets | Predicting outcomes | Aggregating judgements using the Internet and Web to predict public and private events | |
| Seriosity | Collaboration through massive multi‐player online games (MMOGs) | Use of MMOG to help prioritize and manage e‐mail and manage information overload generally | |
| 3.0 Co‐creating | Firefox | Open source software development | Prioritization of features to produce a more user‐friendly version of the Mozilla browser |
| Simple Wikipedia | Open ‘wiki’ content creation, allowing users to collaborate by adding and editing online content | Writing, simplifying complex text entries in Wikipedia | |
| A Swarm of Angels | Open production of creative artefacts, such as a film | International creator‐led collaborative development of all aspects of making a film |
We chose our cases carefully, with each reflecting a unique class of similar projects, while together they cover a wide spectrum of types according to the classification dimensions. For instance, we looked at ‘Simple Wikipedia’ to reflect a wide range of wiki projects.
Our typology illustrated in Table 3 can be more concretely understood by a brief outline of each of our case studies, showing how these features of the collaboration networks are utilized. Each case is discussed within the context of the classification we have used to define its central architecture.
Collaboration 1.0 Cases: Hypertextually Shared Documents, Data and Objects
1. Designing and Managing the Atlas Project16
Much scientific collaboration is increasingly distributed, but we chose to focus on an extreme case of collaboration with Atlas: a project launched in 1992 that engages nearly 2,000 scientists in the design of a large‐scale high‐energy physics (HEP) detector facility.17 Scientists travel to CERN frequently for face‐to‐face meetings, but core aspects of collaboration among 165 working groups distributed across the world have been managed for over a decade through the use of e‐mail, attachments, listservs, and shared Web‐based documents. CERN was where the World Wide Web was invented, and this platform has become a central infrastructure for sharing documents among its distributed teams of scientists. There are also other major applications of advanced Internet technologies, primarily the CERN Grid, that are designed to support shared computing facilities for Atlas researchers.
2. Bugzilla:18 Managing the Repair of Software Bugs
A study of Bugzilla gave an insight into open source software development, focusing on the identification and management of the repair of software bugs in Firefox, one of the principle software projects within Mozilla's collection of open source software projects.19 At the core of Bugzilla is a shared database that helps to coordinate the work of a distributed array of individuals who wished to contribute to the software by either notifying Mozilla of the defects, or contributing to their repair. However, the value of Bugzilla is anchored in a shared document system.
3. InnoCentive:20 Competing for Prizes and Generating Solutions for Users
InnoCentive is one of the more successful networks offering a prize as an incentive for individuals or groups to solve problems. By matching researchers (‘problem‐solvers’) to companies with problems (‘problem‐seekers’), it is used primarily to broadcast problems as a means of finding problem‐solvers rather than as a medium for collaborative work. It also employs the full potential of the Internet to find and match solvers with seekers. Our research also drew on studies by others. InnoCentive was included because it represents a distinctive approach to incentivizing participation.21
4. NeuroCommons:22 Opening Access to a Biomedical Information Commons
NeuroCommons, which is part of the Science Commons, is a project in the medical and pharmaceutical area that typifies a priority within the broad open source movement: to enable open searches of the content of databases, not simply the title of scientific articles. This enables scientific access to specific information that might otherwise be invisible on the Web. With NeuroCommons, users can access multiple datasets to address diverse sets of problems. This represents an evolution towards a semantic Web in which machines will be able to distinguish content based on its meaning in different contexts.
Collaboration 2.0 Cases: Communicating User‐Generated Content
5. Digg23 and Other News Platforms: Aggregating News Content
Originating with Digg, this study evolved into a more comparative survey of news aggregator platforms, focusing on user‐generated and user‐contributed content.24 The value of these networks lies in the way users rate, tag, recommend, view, and comment on news stories, so these activities are central to their performance. Most news aggregation sites are open (e.g. reddit, deli.icio.us, StumbleUpon, in addition to Digg), although they normally require individuals to register before participating in an active way. Stories can receive positive as well as negative votes. By aggregating these ratings, aggregating services can be a help to highlight stories of interest to the larger community of news readers.
6. Sermo:25 Sharing Medical Insights, Information, and Opinions
Sermo is an exemplar of Web 2.0 developments because it offers user‐generated content. This is anchored in the community of licensed physicians in the USA, enabling them to ask questions of one another, post replies, and answer and create surveys.26 In addition, pharmaceutical firms, insurance companies, government agencies, or other potential problem‐holders can pay to see the answers to questions. Third‐parties can also post questions for physicians within Sermo. Physicians remain anonymous but their answers can be rated, creating a reputation assessment for each participant.
7. Information Markets: Predicting Outcomes
The desire to draw on a wide range of expertise to make predictions has been one driving force behind the renewed attention to distributed collaboration. This case reviewed the performance of such information markets, in which individuals are asked to rate the likelihood or probability of different events or outcomes.27 The aggregation of such individual judgements yields a group opinion that is claimed to be more reliable than a single expert under appropriate conditions, such as when the contributors are not prejudiced in trying to sway the outcome.
8. Seriosity:28 Multi‐Player Games for Multi‐Player Collaboration
Seriosity uses the model of a massive multi‐player online game to create incentives for individuals to pay closer attention to their use of e‐mail and in solving their problems with information overload.29 It enables individuals to simulate the redistribution of resources in sending and receiving e‐mail in ways that will lead them to be more strategic about the mail they send and open. There is no group‐shared product, only the potential for individuals—and therefore the organization as a whole—to better allocate their attention to different e‐mail messages.
Collaboration 3.0 Cases: Co‐creating Information through Collaborative Work
9. Firefox:30 Open Source Software Development
Open software production is a key example of co‐creation. Firefox is an open source Internet browser that evolved from the pioneering Mosaic browser, commercialized as ‘Netscape’. When Netscape was eclipsed by Microsoft's Internet Explorer, the Mozilla foundation was created to support its continued development as an open source software project, through collaborative contributions from a distributed network of coders.31
10. Wikipedia:32 Simplifying the Creation of Text in Open Content Creation
The open content Wikipedia online encyclopaedia is among the best known products of distributed collaboration, with a global array of contributors co‐creating a resource that has been compared favourably to any leading encyclopaedia. One of our case studies focused on efforts by the Wikipedia team to simplify the text of selected Wikipedia entries,33 with ‘readability’ scores used to automatically identify ‘unsimple’ entries. The collaboration of individuals in the construction of these entries is a prime example of collaboration 3.0, even though everyday use of Wikipedia by Internet users relies mainly on simple access to shared hypertext documents (collaboration 1.0).
11. A Swarm of Angels:34 Creating a Film
A Swarm of Angels is based in the UK but uses its open source model of movie making to include, from anywhere in the world, distributed collaborators who pay a small fee (£25) to join the production.35 The Director of the project assumes the role of a ‘benevolent dictator’ but enables the community to be polled on controversial issues. Shared information is central in the co‐production process, as are discussion groups and polling.
Overview of Key Themes Emerging from the Case Studies
The study of individual cases and their comparison led to a set of empirically‐anchored themes. These are outlined in the subsections below. Following this overview, three sections expand on particularly significant themes among this set: common characteristics of CNOs; the crucial role of managing the networks; and measuring the performance of collaborative networks.
Reconfiguring Access
The potential of CNOs to create new socio‐technical organization forms stems from the role played by the Internet and related ICTs in ‘reconfiguring access’.36 Use of the Internet can reconfigure the way we do things, such as how we get information, how we communicate with people, how we obtain services, and how we access technologies. Perhaps more fundamentally, it can also alter the outcomes of these activities. It changes what we know, whom we know, with whom we keep in close touch, and what know‐how we require to use the services and technologies to which our access has also been reconfigured. In addition, ICTs—from the printed book to the Web—can change cost structures, by expanding or contracting the geography of access37 and by eliminating or introducing new gatekeepers. Significantly for CNOs, they can also reconfigure access by giving greater or lesser control to users, viewers, or readers.
Importance of the Concept of Collaborative Network Organizations
The Internet is a network of networks that enables organizations and individuals to reconfigure the links between information and individuals across time and space. The concept of ‘network organization’ is therefore useful because it helps to distinguish this emerging organizational form from more place‐based or formal organizations. Although it could be argued that all networks are organizations,38 the concept of a CNO is used here to distinguish the degree to which these are dynamic and inter‐organizational configurations of individuals—as opposed to more institutionalized organizational entities, such as a firm. This does not exclude the potential for CNOs to be created by formal organizations. However, in such instances the networking of individuals achieved is likely to vary significantly from the formal organization.
Commonalities as Well as Diversity
The diversity of these cases, within and between categories, was one of the major observations gained from the case studies. Nevertheless, several commonalities that are central to all distributed collaboration networks were also identified, as discussed below.
The Wisdom of Managing Access over Collaborative Networks
One of the most interesting overall findings questions the notion that CNOs tap ‘the wisdom of crowds’. Instead, the wisdom of these networks lies primarily in the intelligence behind their management, with the contributions of individuals and expertise channelled towards either predetermined specific goals or wider meta‐goals. The cases uncovered a variety of network management levers that can yield more useful outcomes. These include the CNO's architecture design, its degree of openness, the controls employed, and the approaches to the management and modularization of tasks. This indicates that the types of management issues raised by these studies need to be addressed by individuals and their organizations if collaborative network innovations are to capture their potential value fully.
Who Captures the Benefits?
A complex distribution of costs and benefits is involved in CNO development and use. As individuals join and choose to contribute to various distributed problem‐solving networks, they could perceive some benefit in two main ways: by gaining a reduction in their costs through participation; or from their act of participation as such, even through something as intangible as entertaining themselves or boosting their reputation. While the benefits of participating can accrue to individuals or to the providers of platforms, additional costs can be borne by their formal organizations. Moreover, CNOs may introduce fundamentally new participation benefits, reduce costs, or alter how individuals assess these benefits and costs. For example, successful CNOs seem to encourage a strong sense of group identity.39
Commonalities across CNO Categories
Locus of Adoption Decisions: Individuals, Organizations and Networks
Given the patterns of decision‐making illustrated by cases of successful CNOs, the decision to participate in collaboration networks is not a top‐down process. Individuals tend to have the key choice. This is similar to the diffusion of personal computers across organizations in the early 1980s. Just as individual managers and professionals decided to bring their own PC into the office, often against their organization's stated policy, individuals are deciding to join CNOs, often without their colleagues' knowledge or direction. For example, licensed physicians often join Sermo unbeknown to their medical practice, as they view it as a personal productivity tool or have had it recommended by a colleague—not because their parent organization or practice mandated its use. Thus, individual performance is often more salient than organizational or institutional performance.
Top‐Down Goals and Bottom‐Up Choices of Participants
The decision to join highly adaptable collaborative networks is generally not a top‐down process. Instead, individuals tend to have the key choice, often against their organization's stated policy and without the explicit approval, knowledge, or direction of colleagues. However, there is strong leadership of most CNOs that sets a top‐down strategy, such as, in the case of Sermo, networking physicians. That said, their success depends on their ability to effect individual decisions to join and participate.
Complex Ecologies of Actors, Goals, and Objectives
CNOs may be key enablers, but do not solely determine the processes or outcomes of distributed collaboration. These are also shaped by management strategies and the decisions and bottom‐up choices of users. These tend to create networks of peers (physicians, coders, physicists) versus networks within, or between, existing organizations. Given such a general absence of an overriding motivation, participation in networks is usually driven by ecologies of actors with a multitude of often highly individualistic motivations. Across the case studies, investigators found distinctions among actors such as: Insiders vs. Outsiders; Contributors vs. Lurkers; Registered (Angels) vs. Non‐Registered; Platform Providers vs. Users; and Sponsors vs. Users. For instance, the ‘crowd’ who keep CNOs vibrant and successful is typically a small minority of ‘core participants’ who represent a majority of the contributions, with several lurkers also benefiting.
What attracts individuals to CNOs varies greatly, such as:
- •
token or symbolic winnings (e.g. Seriosity) as being potentially as effective as real prizes and payoffs (e.g. InnoCentive or Sermo);
- •
market structures created for some networks (e.g. prediction markets) but not for others (e.g. Atlas);
- •
simulations and games central to some (e.g. Seriosity) but not to others; and
- •
widely varied personal motivations, ranging from ‘Zealots’ to ‘Good Samaritans’ (e.g. Digg or Sermo).
Implications for Distinctions between Problem Holders and Problem Solvers
Traditional distinctions between problem‐holders and problem‐solvers are being blurred, reflecting both the diversity of actors and the degree to which all forms of the new CNOs enable users to be producers themselves. InnoCentive makes an explicit distinction between researchers as problem‐solvers and companies as problem‐holders. Yet this breaks down in practice across most other cases, as the Internet enables individuals to move seamlessly from one role to the other. For instance, readers of Wikipedia, as problem‐holders, can become problem‐solvers when they decide to correct or add to an entry.
The Adaptability and Institutionalization of Networks
CNOs are in continuous evolution. As platform developers and users experience problems, it is usually possible to introduce new forms of moderation, new points of control, and complementary media to resolve issues and maintain the network. Of course, all organizations can evolve new management mechanisms, but some information technologies are viewed as ‘electronic concrete’40 because of the difficulty of making changes to them. While capable of adapting over time, CNOs are likely to face greater problems in sustaining their activity as they compete with more institutional actors, such as the firm.
The Complexity of Evaluating the Performance of Intangible Tasks
The cases examined tended to focus on the use of distributed collaboration for the asynchronous production of intangible information products and services—votes, opinions, ratings, answers, text, designs, software, etc. They do not generally deal with material products. This is because geographically distributed teams also tend to work best asynchronously and the production of intangible information goods and services is increasingly central to many economies. It is true that physical artefacts as varied as a motorcycle or particle accelerator can be designed by distributed teams, but it would be much more challenging to construct one remotely. It is also possible for large, distributed groups to work together on problems involving material outputs, such as the Telegarden41 project that enabled a virtual community to help cultivate a garden. However, asynchronous production of intangible goods appears to enable distributed groups to capture the value of global collaboration, while minimizing the risks associated with mistakes that cannot be corrected, such as flowers and other plants that do not get water.
Table 4 lists some of the performance and evaluation indicators that emerged from the cases. These can be difficult to quantify because of the intangible nature of the products. However, it is clear that different performance criteria need to be developed for different types of CNO.
| Performance being monitored | Illustrative alternative outcomes |
|---|---|
| Susceptibility to threats, error or related risks | Trusted outcomes—or susceptibility to preconceived prejudiced views, gaming of the system, and openness to mob rule |
| Speed | Faster—or ‘more haste, less speed’ |
| Quality of information | Improving accuracy—or dumbing down |
| Information sharing, transfer | Piecing together of an information puzzle to reveal patterns and new insights—or overload and distraction of participants |
| Control over information | Security—or loss of sensitive or proprietary information |
| Agenda‐setting | Responding to set agenda—or enabling users to change focus and shaping of users' attention |
| Independence from company, organizational, institutional bias | More independent information—or less relevant work, undermining organizational objectives |
The Wisdom of Controlling Crowds
Analysis of our case studies showed that the CNOs investigated are not ‘tapping the wisdom of crowds’. Instead, each platform manages the contributions of individuals and expertise in ways that contribute to pre‐determined designs, be they specific goals or more loosely defined meta‐goals. The wisdom of these networks therefore seems to be located in the intelligence behind their management. The providers can shape the patterns of behaviours and norms of use for their networks in ways that yield useful outcomes through a variety of management levers. Table 5 suggests some of the key ways in which CNO platforms can manage contributions to encourage constructive contributions.
| Mechanism | 1.0 Sharing | 2.0 Contributing | 3.0 Co‐creating |
|---|---|---|---|
| Architecture | One‐to‐Many | Many‐to‐Many | Many‐to‐One |
| Openness | Open | Networked | Managed |
| Control | Low | Moderate (reputation) | High |
| Modularization | Low | Moderate (simple tasks) | High |
Determining the Architecture of the Network—and Who Participates
Deciding on the type of architecture used, as in our typology framework, is a crucial step platform managers can take to shape performance and outcomes. Another key point of control relates to who participates in the network and the degree of openness adopted (e.g. the wide‐open access to readers of Wikipedia; Sermo's initially limited access only to registered physicians in the USA; or the request by A Swarm of Angels for contributors from anywhere to pay a modest fee).
It is also possible to have tiered levels of access to different elements of the application. For example, Wikipedia managers can: close an entry, thereby closing off editing completely; limit access by allowing trusted members of the community to resolve editorial issues; or give some trusted contributors the permission to delete the work of others. Most networks create a hierarchy of rights and privileges that determine who can do what within the network, enabling them to configure access to key resources in numerous ways. The need for this type of control over access, including tiering, is greater in the 3.0 networks than the 1.0 networks.
The management structures of various networks vary, but several have more hierarchical than egalitarian arrangements for handling peer production.42 Some, like A Swarm of Angels, are managed by a self‐announced ‘benevolent dictator’. Others, such as Atlas, have sought to support peer review and consensual decision‐making, but permit leadership to evolve within teams and workgroups. In contrast, even in the several CNOs representing bottom‐up communities (e.g. Sermo or Seriosity), there is usually a core authority responsible either for membership into the community or a core principle governing how the community ‘plays the game’ or interacts.
Finally, all the cases indicate that CNO platforms employ mechanisms to simplify tasks to make them manageable for individual problem‐solvers and problem‐holders. Given the complex array of often personal motives behind individual participation in these networks, the cost of participation must be kept low. One major strategy in this area is to modularize the product in ways that do not overwhelm contributors. Wikipedia can ask contributors to edit single entries by making incremental changes and additions. Bugzilla modularizes the repair of defects into precise software projects. Sermo offers numerous mechanisms to simplify the contributions of physicians, such as encouraging their input to be provided in the form of answers to multiple choice questions.
Assessing the Performance of CNOs
Differences across the case studies highlighted many issues of performance surrounding CNOs. This understanding of major differences across CNOs makes it important to distinguish the critical points of performance tied to each type of network, such as:
- •
1.0 Sharing: how widely a network's hypertexted information is read, cited, and rated highly. Who finds it helpful? What are its in and out links? Answers to such questions can be assessed to a degree by log files identifying those who go to a Web page. However, most log files of sites are confidential and not publicly accessible, although the providers of Web‐based information can assess the level and range of interest in their information over time and make comparative judgments of its value. Log files can lead to follow‐on qualitative interviews with, or surveys of, users.
- •
2.0 Contributing: the key performance issues here relate to the degree to which a CNO draws actors to make contributions (e.g. answers, ratings, votes). The rating of books on Amazon.com is claimed to be evidence of the site's success because books are often rated by many individuals. Sermo appears to be relatively successful because many (over 50,000) licensed physicians register to use the system, and significant sub‐groups of physicians participate in surveys.
- •
3.0 Collaboration: a key indicator of the performance of a CNO in this category is its ability to attract and sustain relevant contributors to the production of information products or services. Wikipedia entries are rated highly not simply because they are written well, but because they attract experts in the respective topics. Such criteria of viewing, contributing, and collaborating are necessary but not sufficient conditions of performance. However, they do help to concentrate attention on those networks that are the most credible sites of high performance.
Webmetrics could be suited to examining the centrality of information within particular networks and gaining a first approximation of the performance of contributions to some collaboration networks.43 However, there are major limitations on Webmetrics, including constraints on access to linking and log data within the platform and the relatively small size of many collaboration networks. Webmetrics proved useful in only one of our case studies: assessing news aggregators.44 Here, the analysis indicated that the most central platforms, such as Digg, tended to occupy a location between traditional news sites and online actors, such as bloggers. This suggests they might play a ‘brokering’ or ‘bridging’ role in the online information environment.
Comparative Indicators of Performance
One difficulty of assessing most uses of ICTs is that new applications often do new things—but seldom the same thing that would be done through other means. For example, prediction markets could be considered as substitutes for expert opinion or survey research, but are very different from both alternatives and are more likely to be used as a complement than a substitute for other sources of information.45
Distribution of Costs and Benefits of Collaborative Networks
Our case studies suggest that the physical and organizational geography of distributed collaborative networks leads to uncertainties over who pays and who benefits. Individuals often make the critical decision on whether or not to contribute to a CNO. The benefits to individuals, the larger network of participants, and any sponsors are likely to be the least problematic for those networks that are successful. However, if these same actors do not perceive a specific collaborative network to be of value, their participation declines and the network fails. The speed with which these networks can be launched and succeed or fail is one critical aspect of their evolution. However, even successful networks could raise reasonable questions about the payoffs to the organizations that employ the participants in these boundary‐spanning networks. Why pay the salaries of individuals spending their time on another organization's project?
One rationale is that if the project supports the performance of individuals within the organization, then the organization as a whole might well capture the benefits. Another is the value of expanding the networks of organizations beyond their boundaries. March46 argued that organizations face a trade‐off between focusing on the search for new ideas through learning and experimentation, versus focusing on the exploitation and refinement of the existing knowledge of an organization. Distributed collaborative networks can enhance both, but the inter‐organizational dimension of most of the networks we studied suggests that they are a particularly useful source of new ideas from outside their existing organizational context.47
The distribution of costs associated with collaborative networks are such that individual contributions are often incremental, and therefore easily absorbed by their home organizations. The design of networks, particularly the modularization of tasks, helps to minimize the costs for a participant. To understand the dynamics of shaping who pays and who benefits, it is important to look at the nature of intellectual property rights (IPR) created. These will vary across collaboration network types, as shown in Table 6, which reinforces the importance of this typology of CNO.48
| Locus of IPR | Cost centre | Benefits capture | |
|---|---|---|---|
| 3.0 Collaboration | Co‐created product | Creation, implementation, management of co‐created product (e.g. software, films) | Licensing or sale of information product; non‐monetary benefits (e.g. training, status, notoriety) |
| 2.0 Contributing | Platform for soliciting and processing contributions | Creation, implementation, management of platform for generating and managing community input | Licensing or sale of the platform or network (e.g. for advertising or third‐party access to the network) |
| 1.0 Sharing | Information being shared, sold, or advertised | Creation of the information product | Authors, creators, aggregators gain reputation, influence, or fees |
Conclusions: Key Dimensions of Collaborative Network Organizations
The vitality of the networks explored through the OII–MTI case studies suggests that CNOs will be of growing importance. Understanding which dimensions of their design, structure and use contributes most to the difference between success and failure is therefore of importance in related practice and research. The cases surfaced the significance of the role of CNOs as ‘networks’ rather than ‘organizations’. CNOs are different from the formal structures defined in organization charts. Their performance depends largely on the choices made by individuals involved, including their ability to participate in corporate or project‐focused networks while choosing to also join many other Internet‐enabled networks within and outside their organization.
Despite their clear potential, these networks do not represent an information Utopia. Many fail, and even the best networks face major management challenges. These pose critical issues to be faced by those who manage existing CNOs, as well as by managers of organizations who might wish to capture the value of these networks without unwarranted risks to their own organization. Overall, these issues fall into three broad categories: managerial, social and technical.
Managerial Underpinnings
The Centrality of Managing Networked Individuals
The case studies have shown the central importance of clear management structures in coping with the challenges of distributed collaboration.49 The rhetoric of the ‘wisdom of crowds’ can deflect attention from the degree to which successful CNOs are best viewed as managed networks of individuals who choose whether and when to enter or exit a network. Crowds are more often unmanaged and accidental. Leadership and management structures play a key role in recruiting participants, maintaining their involvement, and managing their contributions. Important factors in managing CNOs include the need to:
- •
Create a critical mass of users to sustain the network. Type 1.0 networks can be open to the world, but many type 2.0 and 3.0 networks need to restrict participation. In order to limit or enlarge the size of a network, the platform's manager can regulate participation in a variety of ways, such as freezing membership or bringing in a new cohort of participants.
- •
Structure tasks, just as individuals might structure their ‘to do list’. Incentive structures and information systems need to address issues surrounding the competition for users' attention. Which pages need editing? Which bugs need to be fixed? Which e‐mails are important to read right now?
Use of CNOs can Support or Undermine Organizations
Managers and professionals need to understand the risks involved in running a CNO, in order to capture the benefits. Our case studies demonstrate that gains from distributed intelligence systems can be great, such as from knowledge sharing, social networking, and collaboration across time and space. The fact that CNOs are not well aligned with the boundaries of formal organizations can also be positive (e.g. in supporting better surveillance of intra‐ or inter‐organizational environments). However, these networks also bring some risks, such as in the loss of control over private, proprietary, or sensitive information.50 Multiple threats also arise from errors (e.g. in prediction cases where small markets could magnify prejudices).
Social Underpinnings
The Challenge of Building a Motivated Ecology of Contributors
The case studies focused on successful CNOs, but many efforts to create networks fail to attract a critical mass of contributors. As technologies for supporting these networks become more accessible and powerful, it will become increasingly important to confront the crucial challenge of envisioning and constructing a strategy for developing an ecology within which all stakeholders have clear ‘wins’. For example, physicians contributing to Sermo value the community or the information they can obtain through the network; the platform provider has found a successful business model; and sponsors perceive a net gain.
Individuals can Lose or Gain by Participation in CNOs
People can choose to participate in distributed collaborative networks and can enter and exit at will. However, like organizations, it is important for individuals to assess their participation critically. Who benefits? Who gains? Carr51 puts it most graphically by arguing that users of Web 2.0 platforms may be becoming a ‘global pool of cut‐rate labor’ for the ‘digital elite’ in the age of the information utilities, such as search engines. In the cases examined here, the most successful demonstrated benefits to a wide range of individuals—not simply their developers.
Technical Underpinnings
Technology Matters
Differences in the underlying technical designs and architectures of networks are important in CNO performance because they determine how networks can be used to reconfigure access to information and people in collaborations. Network types 1.0 (sharing), 2.0 (contributing), and 3.0 (co‐creating) identified in this paper are overlapping and general, but illustrate some fundamental differences in design that have shaped how these collaborations work in reconfiguring information flows within and across real and virtual organizations.
Solutions Looking for Appropriate Problems
A suitable network does not follow simply once a problem has been recognized. Many Internet‐based platforms and related ICTs represent solutions for addressing a vast range of problems; however, not all problems appropriately align with such technologies. One issue is the legitimacy of a problem. For instance, some potential applications in prediction markets have been judged to be ethically questionable (e.g. asking people to predict a human tragedy, such as an assassination). The need to simplify tasks to foster participation (e.g. creating an easily‐answered questionnaire) can place practical limits on the quality of information obtained. It is also unclear whether some collaborative networks, such as Type 2.0, can handle complicated, interdependent tasks.
Contributions to Further Studies on Collaborative Networks
These themes were developed from an array of case studies. It is important to emphasize that the cases formed the basis of a qualitative study aimed at drawing from different forms of distributed problem‐solving networks. They are not representative of CNOs, or even successful ones. However, these themes should give direction to further research. If reinforced by other studies of other networks, our findings could contribute to a more critical perspective on the ‘wisdom of crowds’ that focuses more attention on the management of networked individuals.