“Crowdsourcing” now gets 10 million hits on Google, and is being applied to research in commercial, media, academic, civil society and state spheres. Although appearing recent and technology-based, there are also relevant but overlooked manual precursors which embody the fundamentals of using large groups for research. This analytical review provides the bases for developing initiatives further, by assessing: What are the strategies, strengths and weaknesses of crowdsourcing research? What are the questions that should be asked when planning a research design? How is crowdsourcing being applied in relation to state crime, and why? What might be the implications for justice systems, and for criminal and international courts?
AIN ( 2012) “Nigeria Amnesty Dialog”, Amnesty International Nigeria . Available online at http://ngamnesty.partners-international.org/ (accessed 29 March 2012).
Andersen, M. ( 2009) “Four Crowdsourcing Lessons from the Guardian's (Spectacular) Expense-Scandal Experiment”, Nieman Journalism Lab . Available online at http://www.niemanlab.org/2009/06/four-crowdsourcing-lessons-from-the-guardians-spectacular-expenses-scandal-experiment/ (accessed 29 March 2012).
BBC ( 2010) “Anonymous Wikileaks activists move to analogue tactics”, BBC News . Available online at http://www.bbc.co.uk/news/technology-12008565 (accessed 27 March 2012).
BBC ( 2012) “Outlook: The Egyptian psychiatrist treating the country's torture victims”, BBC News . Available online at http://www.bbc.co.uk/podcasts/series/outlook#playepisode5 (accessed 27 March 2012).
Beissel-Durrant, G. ( 2004) “A Typology of Research Methods within the Social Sciences”, Unpublished Working Paper. ESRC National Centre for Research Methods . Available online at http://eprints.ncrm.ac.uk/115/ (accessed 26 March 2012).
Berdou, E. (n.d.) Mediating Voices And Communicating Realities: Using Information Crowdsourcing Tools, Open Data Initiatives, and Digital Media to Support and Protect the Vulnerable and Marginalised . London/Brighton: DFID/IDS.
Berk, A. ( 1974) Collective Behavior . Dubuque, Iowa: Wm. C. Brown.
Bernstein, M.S. et al. ( 2010) “Soylent: A Word Processor with a Crowd Inside”, UIST . Available online at http://groups.csail.mit.edu/uid/other-pubs/soylent.pdf (accessed 2 April 2012).
Booth, W. ( 2010) “Mayor of Cancun, Mexico, charged with drug trafficking, money laundering”, Washington Post , 27 May.
Bytwerk ( 2012) “German propaganda archive”, Bytwerk . Available online at http://www.bytwerk.com/gpa/posters2.htm (accessed 27 March 2012).
Catone, J. ( 2007) “Outsource Your Brain for Science”, New Scientist , 12 July.
CCR ( 2011) “Clergy Victims File International Criminal Court Complaint: Case Charges Vatican Officials with ‘Crimes Against Humanity’”, Centre for Constitutional Rights . Available online at http://www.ccrjustice.org/iccvaticanprosecution (accessed 8 April 2012).
Cosmides, L. and Tooby, J. ( 1992) “Cognitive Adaptations for Social Exchange”, in J.H. Barkow, L. Cosmides and J. Tooby, eds, The Adapted Mind: Evolutionary Psychology and the Generation of Culture . New York: Oxford University Press.
Crowdsourcing.org ( 2012) “Community tools”, Crowdsourcing.org . Available online at http://www.crowdsourcing.org/community/tools/8 (accessed 30 March 2012).
CWDLTP ( 2012) “Civil War Diaries & Letters Transcription Project”. Available online at http://digital.lib.uiowa.edu/cwd/transcripts.html (accessed 29 March 2012).
de Castella, T. ( 2010) “Should we trust the wisdom of crowds?” BBC News . Available online at http://news.bbc.co.uk/1/hi/magazine/8788780.stm (accessed 3 April 2012).
EI ( 2012) “Syria's revolution finds a temporary home in Cairo”, Egypt Independent . Available online at http://www.egyptindependent.com/node/657146 (accessed 2012).
El Nadeem Centre ( 2006) Torture in Egypt: A State Policy . Cairo: El Nadeem Centre.
Estellés Arolas, E. and González Ladrón-de-Guevara, F. ( 2012) “Towards an Integrated Crowdsourcing Definition”, Journal of Information Science (in press).
Fisk, R. ( 2001) “Death in Bethlehem, made in America”, The Independent , 15 April. Available online at http://www.middleeast.org/read.cgi?category=Magazine&num=345&standalone=&month=8&year=2001&function=text (accessed 27 March 2012).
Foucault, M. ( 1975) Surveiller et punir: Naissance de la prison . Paris: Gallimard.
Foxe, J. (ed. Williamson, G.A.) ( 1965) Foxe's Book of Martyrs . London: Secker and Warburg.
Fraser, R. ( 1990) The Making of The Golden Bough: The Origins and Growth of an Argument . London: Palgrave Macmillan.
Frazer, J.G. ( 1993) The Golden Bough: A Study in Magic and Religion . London: Wordsworth Editions.
Freeman, T.S. ( 2004) “Foxe, John (1516/17–1587)”, in Oxford Dictionary of National Biography . Oxford: Oxford University Press.
Geyer, F. and van der Zouwe, J. ( 1992) “Sociocybernetics”, in C.V. Negoita, ed., Handbook of Cybernetics . New York: Marcel Dekker.
GPPW ( 2010) “Polluter Watch”, Greenpeace . Available online at http://www.polluterwatch.com/bp-documents (accessed 1 April 2012).
Green, P. and Ward, T. ( 2004) State Crime: Governments, Violence and Corruption . London: Pluto Press.
Guardian ( 2009) “Investigate your MP's expenses”, The Guardian . Available online at http://mps-expenses.guardian.co.uk/ (accessed 3 February 2010).
Hawk, D. ( 2003) The Hidden Gulag: Exposing North Korea's Prison Camps . Washington: US Committee for Human Rights in North Korea.
Hawley, C. ( 2010) “Drugs cast cloud over Mexican paradise”, USA Today , 26 May.
HCL ( 2012) “Harvard Tehran propaganda murals”, Harvard College Library . Available online at http://hcl.harvard.edu/collections/digital_collections/tehran_murals.cfm (accessed 27 March 2012).
Heingartner, D. ( 2003) “Back together again”, New York Times , 17 July. Available online at http://www.nytimes.com/2003/07/17/technology/back-together-again.html (accessed 27 March 2012).
Howe, J. ( 2009a) Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business . New York: Crown Business.
Howe, J. ( 2009b) “The Return of Crowdsourced Border Control”. Available online at http://www.crowdsourcing.com/cs/2009/02/the-return-of-crowdsourced-border-patrol.html (accessed 18 May 2012).
Hudson-Smith, A. ( 2011) “Crowdsourcing Opening Up New Opportunities”, Methods News . London: ESRC.
IBC ( 2012) “Documented Civilian Deaths from Violence”, Iraq Body Count . Available online at http://www.iraqbodycount.org (accessed 27 March 2012).
iRevolution ( 2012) “Crisis Mapping Syria: Automated Data Mining and Crowdsourced Human Intelligence”, iRevolution . Available online at http://irevolution.net/2012/03/25/crisis-mapping-syria (accessed 28 March 2012).
ISCI ( 2012) “Arab Spring: The Seeds of Change”. Unpublished conference discussion, 23 March. International State Crime Initiative, King's College London.
King, J.N. ( 2006) Foxe's Book of Martyrs and Early Modern Print Culture . Cambridge: Cambridge University Press.
Kittur, A. et al. ( 2009) “Crowdsourcing for Usability: Using Micro-task Markets for Rapid, Remote and Low-Cost User Measurements”. Available online at http://www.clickadvisor.com/downloads/Kittur_2009_Crowdsourcing_for_Usability.pdf (accessed 2 April 2012).
LaFraniere, S. ( 2012) “Activists crack China's wall of denial about air pollution”, New York Times , 28 January.
Lintott, C. ( 2012) “Volunteers wanted for planet hunt”, BBC News . Available online at http://www.bbc.co.uk/news/science-environment-16512017 (accessed 29 March 2012).
LSE ( 2008) “Long wave events”, MacKinder Programme . Available online at http://www.lse.ac.uk/collections/mackinderCentre/ (accessed 2 April 2012).
MacGregor, C. ( 2009) “Innovation: The Sinister Powers of Crowdsourcing”, New Scientist . Available online at http://www.newscientist.com/article/dn18315-innovation-the-sinister-powers-of-crowdsourcing.html (accessed 5 April 2012).
Masami, N. ( 2004) “Bikini: 50 years of Nuclear Exposure”, Japan Focus . Available online at http://japanfocus.org/-Sadamatsu-Shinjiro/2121 (accessed 1 April 2012).
MCSY ( 2012) “Media Centre of Syrian Revolution”, Facebook . Available online at http://www.facebook.com/MCSYRIA9 (accessed March 2012).
Milmo, C. ( 2011) “Lloyd's insurer sues Saudi Arabia for ‘funding 9/11 attacks’”, The Independent , 19 September.
Monbiot, G. ( 2010) “These Astroturf libertarians are the real threat to democracy”, The Guardian , 14 December.
Moses, A. ( 2009) “Amateur spies put North Korea on the map”, Brisbane Times . Available online at http://www.brisbanetimes.com.au/technology/amateur-spies-put-north-korea-on-the-map-20090602-bttj.html (accessed 4 April 2012).
MSFLI ( 1980) “Dānishjūyān-i Musalmān-i Payraw-i Khatt-i Imām, Dānishjūyan-i Musalmān-i Payraw-i Khatt-i Imām” (Documents from the US Espionage Den. Published by Muslim Students Following the Line of the Imam). Reconstruction available online at http://www.gwu.edu/∼nsarchiv/nsa/publications/iran/irdoc.html (accessed 27 March 2012).
MT ( 2012) “History of Madame Tussauds”. Available online at http://www.madametussauds.com/London/About/History/Default.aspx (accessed 8 April 2012).
Namba, K. ( 2011) “Light Continues to be Shed on Questions Surrounding Bikini Incident”, Hiroshima Peace Media Centre . Available online at http://www.hiroshimapeacemedia.jp/mediacenter/article.php?story=20110331134439684_en (accessed 1 April 2012).
NDSU ( 2012) “Korean war propaganda leaflets”, North Dakota State University . Available online at http://library.ndsu.edu/digital/korean-propaganda/ (accessed 27 March 2012).
NKEW ( 2009) “North Korea Uncovered”, North Korean Economy Watch . Available online at http://www.nkeconwatch.com/north-korea-uncovered-google-earth/ (accessed 1 April 2012).
Norcie, G. ( 2011) Ethical and Practical Considerations for Compensation of Crowdsourced Research Participants . Pittsburgh: Carnegie Mellon University.
ONI ( 2012) “Iranian Nuclear Sites and other Crowdsourcing GIS News”, Open Nuclear Iran . Available online at http://apb.directionsmag.com/entry/iranian-nuclear-sites-and-other-crowdsourcing-gis-news/230235 (accessed 7 April 2012).
OU ( 2012) The Great War Archive, University of Oxford . Available online at http://www.oucs.ox.ac.uk/ww1lit/gwa (accessed 26 March 2012).
Petrossian, F. ( 2009) “Iranian officials ‘crowd-source’ protester identities”, Global Voices Online , 27 June. Available online at http://globalvoicesonline.org/2009/06/27/iranian-officials-crowd-source-protester-identities-online/ (accessed 6 April 2012).
PH ( 2012) “Planet hunters”, Zoo Universe . Available online at http://www.planethunters.org/ (accessed 30 March 2012).
PRM ( 2012) “What are field collectors?” Pitt Rivers Museum . Available online at http://history.prm.ox.ac.uk/page_74.html (accessed 12 April 2012).
Quinn, A.J. and Bederson, B.B. ( 2009) “A Taxonomy of Distributed Human Computation”, University of Maryland, Department of Computer Science . Available online at hcil.cs.umd.edu/trs/2009-23/2009-23.pdf (accessed 27 March 2012).
Quinn, A.J. ( 2011) “Human Computation: A Survey and Taxonomy of a Growing Field, CHI 11”, University of Maryland, Department of Computer Science . Available online at http://www.efiko.org/material/Human%20Computation-%20A%20Survey%20and%20Taxonomy%20of%20a%20Growing%20Field%20by%20Alexander%20J.%20Quinn.pdf (accessed 30 March 2012).
Sarhandi, D. ( 2001) Evil Doesn't Live Here: Posters from the Bosnian War . New York: Princeton Architectural Press.
SBTF ( 2011) “Combining Crowdsourced Satellite Imagery Analysis with Crisis Reporting: An Update on Syria”, Stanby Task Force . Available online at http://blog.standbytaskforce.com/combining-crowdsourced-satellite-imagery-analysis-with-crisis-reporting-an-update-on-syria/ (accessed 29 March 2012).
Scher, D.L. ( 2012) “Crowdsourced Clinical Studies: A New Paradigm in Health Care?” Available online at http://davidleescher.com/2012/03/30/crowdsourced-clinical-studies-a-new-paradigm-in-health-care/ (accessed 15 June 2012).
Schmidt, L. ( 2011) Crowdsourcing for Human Subjects Research . Oakland: HeadLamp Research.
SNAP ( 2012) “International Law and World Class Action Lawsuit Against the Roman Catholic Church”, Survivors Network of those Abused by Priests . Available online at http://www.worldlawdirect.com/forum/international-law-issues/52945-international-law-world-class-action-lawsuit-against-roman-catholic-church.html (accessed 3 April 2012).
SPGP ( 2008) “Get involved”, Sentinel Project for Genocide Prevention . Available online at http://thesentinelproject.org/get-involved-2/volunteering-with-the-sentinel-project/ (accessed 30 March 2012).
SSP ( 2010) “Monitoring the Crisis in Sudan”, The Satellite Sentinel Project . Available online at http://satsentinel.org/ (accessed 30 March 2012). (Twitter – @SudanSentinel).
Surowiecki, J. ( 2004) The Wisdom of Crowds: Why the Many are Smarter than the Few and how Collective Wisdom Shapes Business, Economies, Societies and Nations . London: Doubleday.
Syria Tracker ( 2012) “Syria crisis map”, Syria Tracker . Available online at http://syriatracker.crowdmap.com/ (accessed 29 March 2012).
Takagi Fund ( 2012) “Citizen Science”, Takagi Fund . Available online at http://www.takagifund.org/e/about/index.html (accessed 1 April 2012).
Theodora ( 2012) World Oil Pipelines . Available online at http://www.theodora.com/pipelines/world_oil_gas_and_products_pipelines.html (accessed 15 June 2012).
TI ( 2012a) “Monitoring the Renovation and Reconstruction of the Mayor's Office via Crowdsourcing”, Transparency International . Available online at http://transparency.ge/en/activity/monitoring-renovation-and-reconstruction-mayors-office-crowdsourcing (accessed 1 April 2012).
TI ( 2012b) “Malaysian Forest Monitoring and Other Crowdsourcing GIS News”, Transparency International . Available online at http://www.timalaysia-forestwatch.org.my and http://apb.directionsmag.com/entry/xx-and-other-crowdsourcing-gis-news/234425 (accessed 3 April 2012).
Tumatauenga, N. ( 2012) “Crowdsource: Unconventional Warfare in Iran”, Milphotos.net . Available online at http://www.militaryphotos.net/forums/showthread.php?207248-Milphotos-net-Crowdsource-Unconventional-Warfare-in-Iran (accessed 27 March 2012).
Turnbull, A. ( 2012) “Alan Turnbull's Secret Bases”, Secret Bases . Available online at http://www.secret-bases.co.uk/google-earth.htm (accessed 27 March 2012).
USDJ ( 1998) From Pain to Power: Crime Victims Take Action . Washington: US Department of Justice. Available online at http://www.ncjrs.gov/ovc_archives/reports/fptp/welcome.html (accessed 15 June 2012).
Vela, J. ( 2012) “Turkey's former president goes on trial for 1980 coup”, The Independent , 5 April.
Weiner, N. ( 1949) Cybernetics: Or, Control and Communication in Man and Machine . Cambridge, Mass.: MIT Press.
Wiki ( 2012) “List of Crowdsourcing Projects”, Wikipedia . Available online at http://en.wikipedia.org/wiki/List_of_crowdsourcing_projects (accessed 26 March 2012).
Wiki-Cancun ( 2012) Cancun airport . Available online at http://en.wikipedia.org/wiki/Canc%C3%BAn_International_Airport (accessed 15 June 2012).
WikiLeaks ( 2012) “Secret US Embassy Cables”, WikiLeaks . Available online at http://wikileaks.org/cablegate.html (accessed 27 March 2012).
Williams, C. ( 2006) Leadership Accountability in a Globalizing World . London: Palgrave Macmillan.
Williams, C. ( 2012) Researching Power, Elites and Leadership . London: Sage.
Winchester, S. ( 1998) The Surgeon of Crowthorne: A Tale of Murder, Madness and the Love of Words . London: Viking.
WWMD ( 2011) “World Water Monitoring Day”, World Water Monitoring Challenge . Available online at http://www.worldwatermonitoringday.org/ (accessed 1 April 2012).
WP ( 2007) “Drawings of Genocide”, Waging Peace . Available online at http://www.wagingpeace.info/index.php?option=com_content&view=article&id=117&Itemid=30.