Omnichannel Heuristics for E-commerce

Many organizations are providing their services via web and apps, however, appropriate methods for measuring usability and user experience of a digital ecosystem seem to be largely lacking. Ecommerce has become popular in developing countries like Nepal and its usefulness is found to be high for example during emergencies like the pandemic. The purpose of this research is to compare one of the mostly used inspection evaluation method i.e. Nielsen’s heuristics with Omnichannel heuristics method which is developed considering the digital ecosystem in the context of e-commerce in omnichannel media.


INTRODUCTION
Since the mid-1980s, there is increased interest in the field of Human Computer Interaction (HCI). Today's ubiquitous nature of computers has added more responsibility to people working in the HCI. Usability has been a core concept in HCI, which has slowly shifted toward user experience (UX). ISO 9241-11 defines usability as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use" (Bevan, 2001). Usability measurement has been an important part of user-centered design (UCD). The term emerged in the early 1980s. The main principle was to keep the user needs at the center of design (Marti and Bannon, 2009). In today's world, technology is changing more rapidly than ever before. People are trying to utilize technology to the fullest to accomplish things in their daily activities. Easy access to the Internet with high bandwidth has not limited its use to desktop but has expanded use via mobile devices. Users can now communicate or access the same service via multiple channels. E-commerce and omnichannel have been hugely utilized by developed countries operating their business. Developing countries like Nepal were found to be trailing behind in mastery of e-commerce. The trend in the Nepal market scenario has been changing. E-commerce has become lucrative platform for entrepreneurs of Nepal as it is found to be profitable for their ventures (Ngudup et al., 2005) and the COVID-pandemic has been a game-changer for e-commerce businesses. Many people have started to use e-commerce due to situational demands in a country like Nepal. Each year new e-commerce enters the Nepal market, mostly with websites and app versions in parallel. It was found that the majority of Nepalese customers face issues when using e-commerce (Vaidya, 2019) which is believed to have much more potentials than what it is offering at the moment by using UCD during development of the e-commerce applications. It was found from interviews conducted with Nepalese UX designers that in most cases the design of websites was transformed into its app versions without doing proper research on it. From interviews, it was also found lack of communication between a website and app development teams leading to inconsistent design affecting seamless experience when switching from web version to app version and vice-versa. Usability and UX evaluation were rarely performed, citing lack of budget, time, and experts to conduct an evaluation.
Research found customers abandon e-commerce due to poor site organization (Sivaji et al., 2011). It is important to retain customers to get more revenue by improving the usability and UX of ecommerce. It was found that little emphasis was given on formal usability within the e-commerce sector compared with mainstream business applications (Sartzetaki et al., 2003;Jach and Kuliński, 2012). It was found that heuristic evaluation is a useful usability evaluation method for e-commerce (Sivaji et al., 2011).

RELATED WORKS
Heuristic evaluation, which is the most used inspection method, is easy to perform, and it is inexpensive and effective to use (Inostroza et al., 2013). Usability problem detection depends upon the expertise of the evaluator and computer professional. The heuristic evaluation recommends a small group of experts rather than a single evaluator to evaluate an interface (Hertzum and Jacobsen, 2001). Jakob Nielsen's heuristics is one of the most used heuristics (Nielsen, 1995;Molich and Nielsen, 1990;Liu, 2008). Nielsen and Molich developed this method focusing on telecom systems. It is often argued that it cannot be directly applicable to other software systems (Harrison and Duce, 2013). Another drawback is that it may not be appropriate to find software/device specific problems. Emerging smart devices usability problems cannot be addressed by traditional ways of thinking. New heuristic evaluation methods are needed to address new context i.e. omnichannel.
Zhang's heuristics were developed to address usability problems of specific devices. It was developed to inspect medical devices and particularly used to identify patient safety through the identification and assessment of usability problems (Zhang et al., 2003). Based on Nielsen's heuristic evaluation and Shneiderman's (Shneiderman's,1998) "eight golden rules," Zhang extended his heuristics, from Nielsen's 10 to 14. Zhang tried to incorporate Norman's "7-stages of actions." These heuristics, however, have the same disadvantage as previous. It is targeted to evaluate particular devices and could not address challenges posed by smart devices.
The above-mentioned heuristic evaluations lack the ability to address technology like mobile devices. Addressing this, Yáñez Gómez, R., et al. developed new heuristic guidelines (Yáñez et al., 2014). It consists of a list of 13 heuristics. It has incorporated every important aspect like pleasant and respectful interaction and privacy in the heuristic list. It has a sub-heuristic list which addresses the specific character of mobile, e.g. minimum input needed, general visual cues, fatfinger syndrome, among others. Gomez's heuristics argued to best fit for mobile device evaluation compared to the previously-mentioned heuristic evaluations. These heuristics are primarily focused on touch screen phones and tablets but has not considered smartphones that we now find in most people's pockets and the concept of multiple channel is missing.
The heuristics of Silva et al., which evolved from Nielsen, included 33 heuristics for evaluating smartphone apps targeted at older adults. Results of these heuristics show that all heuristics on the list were used in the evaluation of a smartphone app. Some of the unanswered questions of these heuristics are if this can be equally applied for desktop and mobile websites (Silva et al., 2015).
The above-mentioned heuristics did not consider a customer journey. Customer journey maps are used for understanding the customers' experience. It refers to a pictorial representation of interaction done by customers, possibly through different media at different times with the organization or service provider (M. S. Rosenbaum et al., 2017). Laia Bonastre and Toni Granollers considered customer journeys in their heuristic during purchase in a single channel, i.e. a website (Bonastre and Granollers, 2014). Nowadays, most e-commerce provides services through both website and app versions. The concept of multiple channels is lacking in the Laia and Toni heuristics i.e. website and an app of e-commerce in a single ecosystem. Multiple channel issues were addressed by Brad Aabel and Dilini Abeywarna for digital health (Aabel and Abeywarna, 2018). This heuristic cannot be directly applied to e-commerce. It also lacks concepts of sociability, privacy, and security, which are important for e-commerce. The purpose of Omnichannel heuristics is to measure the user experience of a website and an app of ecommerce considering it as a single ecosystem. As Nielsen's heuristics is the mostly used heuristics evaluation, it is interesting to explore Nielsen's heuristics effectiveness for evaluating omnichannel context for e-commerce and compare results with Omnichannel heuristics. The Omnichannel heuristics were developed by reviewing previous work done to evaluate web and app, output of workshop conducted in Nepalese software companies, feedback from academics and Software companies working on UX, a case study of different e-commerce based on Nepal like "Daraz", "e-Sewa", "Hamrobazar", "IME Pay", "Khalti", and "NETTV", surveys with Google Docs users and The Buddha Air customers, Interviews with Daraz and e-Sewa customers and Interviews with UX designers, UI designers, developers, and Quality Assurance (QA) people.

RESULTS
Three evaluations were conducted to compare the two heuristic evaluation methods. The evaluators were software developers, designers, UX engineers, and UX designers from Nepal based software companies. Different sets of five evaluators were used to evaluate "Buddha Air" using Nielsen's heuristics and Omnichannel heuristics. Three scenarios were used for evaluation. Scenario one was related to flight information and flight status, scenario two to flight routes, and scenario three to booking a flight. Similarly, three evaluators were used to evaluate "Foodmandu" and four scenarios were used for it. The first scenario was related to food order, the second with payment, the third with food delivery, and the fourth was related to listing new restaurants in Foodmandu. Five evaluators were used to evaluate "Daraz" using four scenarios. The first scenario was related to the registration process, the second scenario with a search for a product, the third scenario with payment, and the fourth scenario with listing purchasing experience in Daraz.
Min-Max normalization was used to normalize usability and UX issues found by using Nielsen's heuristics and Omnichannel heuristics. Its objective was to find average value of the issues and compare them. A Comparison between the two heuristics was also done by the priority given on each issue by evaluators. Where P1 represents low priority issue and P4 represents high priority issue respectively.   Comparing evaluators' comments from three evaluations it was found Nielsen's heuristics were unable to address issues related to channel strength and seamless experience, which were addressed by Omnichannel heuristics. Omnichannel heuristics addressed both pragmatic and hedonic issues whereas Nielsen's heuristics only addressed pragmatic issues. Nielsen's heuristics missed the concept of collaborative work and the importance of brand value.
For full details evaluators comment refer https://sameerkharel.wixsite.com/nepal/evaluatorscomments Note: Heuristics in table 5 in the same row reflects similarities between two heuristics. Table 5 shows that Nielsen's heuristics do not say anything related to channel strength whereas as Omnichannel do not say much about "Aesthetic and minimalist design". The concept of collaboration and emotional brand attachment has not been addressed by Nielsen. Sociability is an important concept for e-commerce which is not addressed by Nielsen's heuristics. Also, "privacy" and "security" have not been addressed in Nielsen's heuristics but is covered in Omnichannel heuristics, which is important for e-commerce. For detail heuristics see https://cfe7ec4e-6c4f-4b08bedb-580ef0a5cba2.filesusr.com/ugd/722a37_62cf4a7b6 033415780da4c722c3a1fc4.pdf.

DISCUSSION
Min-Max normalized values from three evaluations using Nielsen's heuristics and Omnichannel heuristics show that 66.67% of Omnichannel heuristics detected more usability and UX issues than Nielsen's heuristics. Three evaluations indicate that both heuristics found common usability and UX issues. Different sets of usability and UX issues were found by Nielsen's and Omnichannel heuristics. Nielsen's heuristics evaluators had less agreement between them compared with Omnichannel evaluators.
Few evaluators felt that Omnichannel is subjective and evaluators might get biased as they might want to prove their skill that may not be the case with Nielsen's heuristics. Some evaluators believed Omnichannel heuristics evaluation takes longer time for evaluation compared to Nielsen's heuristics. Few suggested that further classification was needed in Omnichannel heuristics. Sometimes lack of freedom is provided by Omnichannel heuristics as evaluators only stick to its subheuristics and are seen to provide less in-depth analysis on a few issues over Nielsen's heuristics.

CONCLUSION
The results from Omnichannel heuristic evaluation show that in most cases more usability and UX issues were detected than with Nielsen's heuristics. The result shows high priority usability and UX issues were found by Omnichannel heuristics compared to Nielsen's heuristics. It was found that in most cases comments written by Nielsen heuristics evaluators but not in Omnichannel heuristics were addressed by Omnichannel subheuristics. The result showed Omnichannel was able to detect both hedonic and pragmatic issues, but Nielsen's lack hedonic issues in most cases. Omnichannel heuristics look promising for addressing issues of channel strength and seamless experience, compared with Nielsen's heuristics.