Information quality analysis of online content

The quality of online content can be an important aspect in determining the success of a freecontent website. Whilst there have been many industry best practices focusing on improving accessibility and usability, there is little literature on comparing content quality to aid practitioners in deciding authoring approaches. This study forms part of a larger body of PhD research provide a framework to aid practitioner user-centred design technique selection.


INTRODUCTION
Delivering informative and accessible online content can help organisations to respond to user needs more effectively.This goal is currently aided by a plethora of industry best practices, however there is little information on how to choose the best techniques for individual contexts.In order to produce a framework to aid practitioners, there is a need to understand the key characteristics of online quality to analyse content before and after the application of techniques.This research will examine possible characteristics based on existing technology acceptance models using an action research approach.

Research aim
To investigate the impact of user-centred design technique selection on the quality of content produced.

Research questions
RQ1. Can the quality of informative and accessible content be measured?
RQ2.What is the relationship between content quality and website success?
RQ3.Does the choice of user-centred design technique impact on the quality of content produced?

Research objectives
RO1.To identify characteristics of informative and accessible online content.

RO2.
To define a research model incorporating the relationship between content attributes and content quality.
RO3.To determine the success criteria for the improvement of online content quality.

LITERATURE REVIEW
Within the user research community, there are many issues with the technique selection process.Practitioners often choose techniques based on their own personal experiences due to their familiarity with the methods rather than the project context (Goodman-Deane et al. 2008;López-Mesa and Bylund 2011), meaning that the techniques chosen may not be the most appropriate for the desired results.In addition, engineers often prefer techniques that match their thinking patterns not those of users (López-Mesa and Bylund 2011), resulting in a mismatch in the cognitive models used with a system.At the organisational level, these effects can be exaggerated by the focus on technical aspects rather than user requirements (Castañeda, Muñoz-Leiva and Luque 2007:393).Within industry as a whole, there can be trends of techniques that are adapted into practice rather than followed (Bødker 1998) due to resource constraints and lack of understanding, leading practitioners to conclude the techniques are wholly ineffective rather than incorrectly applied.This can negate the positive aspects of proven techniques.All of these aspects can lead to unsuccessful projects that are delivered late, over-budget and a poor match for user needs.

Online content quality
The WCAG 2.0 guidelines (W3C ( 2008)) define two main categories of content: textual and non-textual.This research aims to study both types of content, with a focus on the parts of online pages that provide meaningful information to the user.The quality of content can have a positive impact on website success (Liu and Arnett 2000), and there are many established measures in industry to help evaluate this.These range from usability guidelines (such as ISO TR 16982 (2002)) to best practice documents (such as W3C ( 2008) and Walker, Black and Carey ( 2013)) to advice on implementing user-centred design practices (such as ISO 9241-210 ( 2010)).These provide varying amounts of guidance to content authors, however there is no definitive answer as to what contributes to 'good' content, and how organisations can improve quality.Rather there is a focus on the way the content is delivered and the structure of the system in which it is contained, leaving specific content creation to non-practitioners who may not have the necessary skills to write for the web.Within the academic literature, this covers three key areas: information quality, usability and user satisfaction.

Information quality
Information quality is an important measure of IS success from a user perspective (DeLone and McLean 1992), and can be used to determine website success (Liu and Arnett 2000).Whilst the main focus of information quality is the perceived usefulness, Castañeda, Muñoz-Leiva and Luque (2007:393) note that users require a 'minimum level of ease of use' in order to evaluate a site's usefulness.This is based on the Heuristic-Systematic Model, where the user focuses on the aspects requiring a minimum effort until they are sufficiently motivated and experienced to not abandon the site.This is confirmed by other studies (Morris and Dillon 1997;Teo, Lim and Lai 1999) that confirm that usefulness depends on ease of use.

Usability
The usability of online systems forms a large part of their quality, and as such many aspects can be drawn from this field as a parallel to content quality.

User satisfaction
User satisfaction is strongly related to the perceived usefulness of an information system, to an extent that when a user perceives a system as providing more value, they are more likely to be satisfied with its use (Mawhinney and Lederer 1990).This has an impact on the information quality, as it demonstrates the importance of user perceptions over empirical measures.Petter, DeLone and McLean (2008:241) note that whilst information quality is inherent in many user satisfaction instruments, it is rarely distinguished as a separate construct, making measurement difficult within IS studies.

Theoretical model
The quality of content will be analysed using the DeLone-McLean model of information systems success (DeLone and McLean (1992), updated DeLone and McLean (2003)).This model defines the impact of a technological artefact on individuals and organisations, based on three aspects of quality: system, information and service.Broadly, these can be considered as corresponding to the three areas of literature drawn upon in this study: 1. System quality As this aspect focuses on the provision of the system itself, it can be considered to encompass the techniques provided by HCI.

Information quality
This aspect focuses on the content provided by the system.

Service quality
The overall quality of the service is parallel to the industry idea of UX (User eXperience), that describes the entire journey of a user through a system.
In their review study of literature based on the application of the DeLone-McLean model to individuals, Petter, DeLone and McLean (2008) identified the strongest influences on information systems success within the model (see Figure 1).This demonstrates that the dominant factor in producing net benefits for individuals is user satisfaction --that is, positive user satisfaction will lead to greater success of the system.In addition, the study demonstrated that the two main factors in producing user satisfaction are the quality of the system and the quality of information within the system.

PROBLEM STATEMENT
Determining the quality of online content is necessary for the evaluation of techniques that may increase accessibility, usefulness and user satisfaction.Many HCI methodologies are metricsbased and evaluate the entire system interface, which can ignore the quality of the content itself.This research will create a set of criteria to aid interpretive information~systems researchers and industry practitioners to determine the relative quality of content before and after the application of information design techniques.

METHODOLOGY
The overall research project is using a Soft Systems Methodology (SSM) approach, as per Checkland (1981).This is appropriate for the activity, as the research involves 'dealing with social or people based project situations' (Gunawardena and Brown 2007:5) with a lack of 'clear or specific objective' (Ho and Sculli 1994:48), requiring 'an enhanced understanding of a complex problem' (Baskerville and Wood-Harper 1996:239).This will provide an opportunity to study information quality criteria in the context in which it is applied (Orlikowski 1992).
The determination of content quality characteristics forms one case study within this project, following Checkland's (1991:26) cycle of action research in human situations.Following this case study, the characteristics will be used to inform the development of content in three more case studies, with individual cycles of improvement within each case.This will provide the opportunity to refine content within each project, applying characteristics individually to evaluate the effectiveness of the application of information quality within a range of contexts.
The researcher has arranged a case study project involving an industry partner, involving the development of content for an information-rich nonprofit website with multiple stakeholder groups to aid in the definition of content quality attributes.The impact of the characteristics on content quality will be improved using a mixed methods approach to measure the information quality in two different ways: 1. Qualitative user surveys A questionnaire based on works such as Kandari et al. (2011a,b) will be used to assess the impact of content changes on users' perceptions.2003) as identified by Petter, DeLone and McLean's (2008) study.

2. Google Analytics data
Anonymised statistics based on user conversions and journeys around the site will be compared to quantify changed usage patterns related to the content.
This double approach is required as 'IQ criteria are often of subjective nature and can therefore not be assessed automatically' (Naumann and Rolker 2005:2), requiring the use of qualitative user feedback to assess the impact of all information quality criteria.These measurements of information quality will be conducted prior to making changes to the website content, and recorded after the changes have been applied to indicate the change in user behaviour.This approach is necessary to show the relative user response to changes to individual aspects within the content.
The initial cycles of research will focus on textual content as this provides meaning to users however they view the websitewhether browsing on a smartphone, tablet, desktop PC or other internetconnected device.As the text will appear on each device, the visual, multimedia, interactive and system aspects (encapsulated by the System Quality and Service Quality elements of the DeLone-McLean model) will not be considered in the survey design as they are outside the scope of this research.The impact of system quality and service quality on systems success has been much explored in the literaure, providing models for the inclusion of such factors in the user-centred design technique selection process.Combined with the initial study, this will provide the basis for the practitioner tool.

MAIN CONTRIBUTIONS
The outcome of this activity will be a set of criteria for evaluating the quality of online content, derived from the relevant information quality literature and the action research study.In addition, a research tool will be created for applying a mixed methods approach to assessing information quality of online websites using incremental improvements.

Q1.
Can a mixed methods approach produce cohesive insights into Information Quality characteristics? Q2.
Is the application of practitioner-generated guidelines a suitable measure of content quality? Q3.
Can content quality criteria be established through a case study approach? Q4.
Can Google Analytics data be analysed to measure the impact of content improvement?

Figure 1 :
Figure 1: Author's illustration of the relative influence of different aspects of the updated DeLone-McLean model (2003) as identified byPetter, DeLone and McLean's (2008) study.