Due to concerns that respondents to online surveys are different from populations of interest, parallel offline surveys can be undertaken and results compared. In this article we create a set of principles to compare results from online surveys with those from surveys using other survey modes. Rather than just comparing estimates and confidence intervals from the different modes, these principles consider biases that each survey mode introduces and whether the results obtained are compatible with each other, given these different biases. Using the example of a survey of platform work, we demonstrate that this approach can be used effectively and be applied to a variety of social science studies that use online surveys.
E. ArmanoA. BoveA. Murgia (eds) (2017) Mapping Precariousness, Labour Insecurity and Uncertain Livelihoods: Subjectivities and Resistance, London: Routledge.
K. BahiaS. Suardi (2019) Connected Society: The State of Mobile Internet Connectivity 2019, London: GSMA.
P. P. BiemerE. D. de LeeuwS. EckmanB. EdwardsF. KreuterL. E. LybergN. C. TuckerB. T. West (eds) (2017) Total Survey Error in Practice, New York: John Wiley & Sons.
M. CallegaroK. L. ManfredaV. Vehovar (2015) Web Survey Methodology, London: SAGE.
E. D. de Leeuw (2008) ‘Choosing the method of data collection’ in E. D. de LeeuwJ. J. HoxD. A. Dillman (eds) International Handbook of Survey Methodology, Hove: Psychology Press:113–135.
E. D. de Leeuw (2018) ‘Mixed-mode: past, present and future’, Survey Research Methods, 12 (2):75–89.
E. de LeeuwN. Berzelak (2016) ‘Survey mode or survey modes?’ in C. WolfD. JoyeT. W. SmithY.-C. Fu (eds) The Sage Handbook of Survey Methodology. London: Sage Publications.
S. DenchR. IphofenU. Huws (2004) An EU Code of Ethics for Socio-Economic Research, Brighton: Institute for Employment Studies.
X. DengR. D. GalliersK. D. Joshi (2016) ‘Crowdworking – a new digital divide? IS design and research implications’, Association for Information Systems Research Papers, 148. https://aisel.aisnet.org/ecis2016_rp/148
V. De Stefano (2016) The Rise of the ‘Just-in-Time Workforce’: On-Demand Work, Crowdwork and Labour Protection in the ‘Gig-Economy’, Geneva: International Labour Office.
D. J. C. DodouF. de Winter (2014) ‘Social desirability is the same in offline, online, and paper surveys: A meta-analysis’. Computers in Human Behavior, 36:487–495.
Eurofound (2015) New Forms of Employment, Luxembourg: Publications Office of the European Union.
A. Felstead (2021) ‘Are online job quality quizzes of any value? Selecting questions, maximising quiz completions and estimating biases’, Employee Relations: The International Journal, 43 (3):724–741.
F. Flanagan (2019) ‘Theorising the gig economy and home-based service work’, Journal of Industrial Relations, 61 (1):57–78.
F. J. Fowler Jr (2014) Survey Research Methods, 5th ed., Thousand Oaks, CA: SAGE.
P. GalvinA. BiermanS. Schieman (2021) ‘Über-alienated: Powerless and alone in the gig economy’, Work and Occupations, published online 22 June 2021.
A. Gooch (2015) ‘Measurements of cognitive skill by survey mode: Marginal differences and scaling similarities’, Research and Politics, July–September:1–11.
A. GreenM. de HoyosS.-A. BarnesB. BaldaufH. Behle (2014) Exploratory Research on Internet-Enabled Work Exchanges and Employability: Analysis and Synthesis of Qualitative Evidence on Crowdsourcing for Work, Funding and Volunteers, European Commission Joint Research Centre Institute for Prospective Technological Studies, JRC Scientific and Policy Reports, Luxembourg: Publications Office of the European Union.
U. Huws (2016) A Review on the Future of Work: Online Labour Exchanges, or ‘Crowdsourcing’: Implications for Occupational Safety and Health, Bilbao: European Agency for Safety and Health at Work.
U. HuwsN. H. SpencerM. CoatesK. Holts (2019) The Platformisation of Work in Europe: Results from Research in 13 European Countries, FEPS – Foundation for European Progressive Studies, UNI Europa, Hatfield: Hertfordshire Business School, University of Hertfordshire.
K. F. Jesnes Braesemann (2019) Measuring Online Labour: A Subcategory of Platform Work, Nordic Future of Work, Brief 2, Oslo: Fafo Institute for Labour and Social Research.
H. JohnstonC. Land-Kazlauskas (2018). Organizing On-Demand: Representation, Voice, and Collective Bargaining in the Gig Economy, Geneva: International Labour Office.
L. F. A. KatzB. Krueger (2016) ‘The rise and nature of alternative work arrangements in the United States, 1995–2015’, NBER Working Paper No. 22667, Cambridge, MA: National Bureau of Economic Research.
J. A. D. KrosnickF. Alwin (1987) ‘An evaluation of a cognitive theory of response-order effects in survey measurement’, Public Opinion Quarterly, 51 (2) 201–219.
I. Mandl (2016) ‘Working conditions in crowd employment’ in W. WobbeE. BovaC. Dragomirescu-Gaina (eds) The Digital Economy and the Single Market, Brussels: Foundation for European Progressive Studies:111–140.
K. MellahiL. Harris (2016) ‘Response rates in business and management research: An overview of current practice and suggestions for future direction’, British Journal of Management, 27 (2):426–437.
C. A. MoserG. Kalton (1971) Survey Methods in Social Investigation, 2nd ed., London: Heinemann Educational Books.
R. O’FarrellP. Montagnier (2020) ‘Measuring digital platform-mediated workers’, New Technology, Work and Employment, 35 (1):130–144.
J. Prassl (2018) Humans as a Service: The Promise and Perils of Work in the Gig Economy, Oxford: Oxford University Press.
N. Rangaswamy (2019) ‘A note on informal economy and ICT’, Electronic Journal of Information Systems in Developing Countries, 85 (3).
J. RossL. IraniM. S. SilbermanA. ZaldivarB. Tomlinson (2010) ‘Who are the crowdworkers? Shifting demographics in Mechanical Turk’, Proceedings of the 28th International Conference on Human Factors in Computing Systems, CHI 2010, Extended Abstracts Volume: 2863–2872.
F. SarracinoC. A. F. RiilloM. Mikucka (2017) ‘Comparability of web and telephone surveys for the measurement of subjective well-being’, Survey Research Methods, 11 (2):141–169.
R. Tourangeau (2018) ‘Data collection mode’ in D. L. VannetteJ. A. Krosnick (eds) The Palgrave Handbook of Survey Research, Cham: Palgrave Macmillan.
P. TubaroC. Le LudecA. A. Casilli (2020) ‘Counting “micro-workers”: Societal and methodological challenges around new forms of labour’, Work Organisation, Labour & Globalisation, 14 (1):67–82.
S. VallasJ. B. Schor (2020) ‘What do platforms do? Understanding the gig economy’, Annual Review of Sociology, 46:273–294.
N. van Doorn (2017) ‘Platform labor: On the gendered and racialized exploitation of low-income service work in the “on-demand” economy’, Information, Communication & Society, 20 (6):898–914.
C. C. WilliamsF. Schneider (2016) Measuring the Global Shadow Economy: The Prevalence of Informal Work and Labour, Cheltenham: Edward Elgar.
A. J. WoodM. GrahamV. LehdonvirtaI. Hjorth (2019) ‘Good gig, bad gig: Autonomy and algorithmic control in the global economy’, Work, Employment & Society, 33 (1):56–75.
J. ZumbrunA. L. Sussman (2015), ‘Proof of a “gig economy” revolution is hard to find’, Wall Street Journal, 26 July.