Agile Search : A tool for searching apps easily on mobile by gesture

Enabling fast access to apps is important to smartphone users especially when there are a large number of apps installed on their devices. Several gesture based search approaches have been proposed to increase the search efficiency and accuracy with improved convenience, but their gesture-app mapping mechanism still largely relies on the textual information of apps. This means such methods can become less effective when a user is trying to locate rarely used apps as they may not be able to recall any textual cues about the app. Since apps are appeared as icons with text descriptions on a device’s screen, there may have other cues that can be used to inform gestures for shortcut access to apps to overcome current text based search/mapping limitations. An initial study involving 24 participants to understand the impact of app icon designs on creating gestures as app shortcuts has been conducted. The results show that users not only tend to use visual cues presented in app icons to create gestures, but also create them based on apps’ usage frequency and different icon types.


BACKGROUND
After the emergence of touch screen device and rapid development of touch sensor technology, using touch gesture to interact with mobile device is becoming a dominant trend.Touch gesture is easy to use; can reducing user's recognize load and extend the input vocabulary; appears in an intuitive way.It is easier to be accepted by users due to, quieter than voice interaction, easier to perform than mid-air gesture, and would not cause spectator's attention.These advantages of using touch gesture to interact have made it more and more popular.
Touch gesture is capable to fulfil many kinds of tasks, and has greatly improved user experience by making users' interaction more lively and interesting.In order to won users' heart, regardless of system functions that integrated in mobile systems (e.g., volume control) and those customized functions within the app all allowing interact action with a gesture.Evidence can be found that Apple and Android, the two giants in the mobile phone market, both enable system touch gesture for interaction in their mobile phone system to meet their users' need.Moreover, under the highly competitive of the app market, many app developers also provide touch gesture support in their app to improve their competitiveness.
Many apps use gesture to serve their user in conduct different tasks, for example game apps use touch gesture to move an object (e.g., the Angry Bird), browser apps enable touch gesture to complete tasks such as switch between different tabs and go "back to" recently visited webpages, tool apps use touch gestures to achieve certain goal, such as the screen-locking apps allow users to use specified gesture as their secret password to lock their mobile screen and so on.
Today, the number of these powerful apps that serve users in an elegant way has begun to shoot up, and this trend has no sign of change.Evidence can be found in the report of enormous number of apps available in app stores.Apple recently reported that there are more than 1.5 million apps (Statista, 2015a) in their App Store with 100 billion app downloads by June 2015 and this number does not appear to be decreasing (Statista, 2015b).
Smartphone has become the hub of people lives.Each iPhone user installs 108 apps on average (Appsfire, 2015), while for Android users, the average app number is at about the same level (95 per smartphone) (Yahoo, 2014).However, a survey showed that on average a user only uses 15 of the installed apps weekly (S.Perez, 2011), and 25 monthly (Krista, 2015).Moreover, users would never revisit over half of apps after its installation (Ofcom, 2014).One of the reasons causing this is that, sometimes it is hard to find the target app, when users want to use a certain app, they cannot reach it quickly in the lengthy app list.In this context, it is necessary to investigated how to help users quickly locate the desired app.

OBJECTIVE AND RESEARCH QUESTION
In my research, the research question is how gesture could be to help users to locate the desired app efficiency.As we all know, users would locate the target app based on their previous experience as well as their recognition of the app.As mentioned earlier, an app can be frequently used or rarely used, and even forgotten, apparently, for these apps divided by the frequency of use, users' effort to locate them are certainly different.Moreover, users would locate the target app based on their previous using apps' experience as well as their recognition of the app.In other words, users have a clear impression of their regularly used app, and the impression is easy to recall, thus enable users to locate the app easily, on the contrary, the less frequently used app leaves an ambiguous impression which is hard for users to recall thus increase the difficulty in locating the app.Some previous research work highlights that, the trouble users have in finding the rarely used apps makes them eventually abandon the already installed apps.But it is not thoroughly studied what effective measures can be taken to overcome the issues are gradually ignored.To fill this gap, our main subject of the research is to learn how to use gesture to ease users' app-locating process regardless of these apps' frequency of use.In order to investigate this main subject, we need to study the following sub objectives:  Understand the current app-locating literature research and existing issues. Investigate the relationship between apps' icon style and user-defined gestures. Propose a user-friendly approach to help them locating target app more efficiency.

LITERATURE REVIEW
Gesture and search are two often studied topics in HCI area.Many researchers studied gesture on the mobile from different angles, such as the perspective of user, gesture usage, recognition algorithm, etc.There are also many studies focusing on understanding search behavior, such as how to improve the hit rate and efficiency of searching, how to improve user's searching experience.Our research focus on using gesture to improve the quality of app-locating process, the next part will be dedicated to the literature review related in this research.

App-locating method
Generally speaking, a target app can be located in two ways, browsing and searching.Taking iPhone as an example where each application screen can only display 20 app icons before grouping, there would need six screens to display 108 apps properly.In that case, a user may need to flip screens back and forth several times to browse the icon lists in order to find an app.
ZapDroid (Singh et al., 2015) was developed to manage infrequently used apps on the smartphone by considering the apps' functionality and use frequency.Moreover Lulu and Kuflik ( 2016) developed a set of menu which helps users to ease infrequently used apps' finding process by cataloguing hints of the corresponding menu (app function, icon color, etc.).Although these approach to some extent reduces difficulty in finding apps, especially for those infrequently used and can somewhat improve browsing efficiency, the process is still relatively slow and when the users' impression of that app is too vague to determine which existing classification this app belongs to, the approaches are useless and would even make things worse.Moreover, Böhmer and Krüger (2013) also found, apps displayed on latter pages of the screen were easier to be neglected, and eventually abandoned by users than those on the front pages.
To address this issue, many works study how to help users to locate target app by searching.Siri of iPhone (Apple, 2015), Search spotlight engine (Aron, 2011), Google now (Google, 2012) in Android, Samsung Sunburst (Samsung, 2011) are all provided user a searching entrance in their system, which can used to random access any items in the mobile by any kinds (e.g., voice, typing in and gesture input) of user given keywords.Compared with browsing, is more efficient, because less pre-steps are needed and the target result would show up directly after user input a keyword.However, its performance relies on (1) whether users can remember any information about the app to form the keywords (recall) and (2) how accurate these keywords are (precision).These methods are efficient when a user can precisely recall relevant textual based information of an app, but when a user wants to find a really used app, while only has vague memory regarding something other than textual based information, these methods are unlikely to be of help.

Search mobile items
Built-in search provides way to help users to locate apps through using keywords.For example, if a user could only remember an app's name contains "p", they still need to browse the result list containing all apps with "p" in their names in order to locate the app they wanted to find.Gesture supported search (Samsung, 2011) has been proposed to tackle this problem by enabling users to map apps with specific gestures.For example, Samsung Sunburst (2011) allows users to use predefined alphabet gestures to launch specific apps.Gesture On supports an always-on gesture based search (Lu, H. and Li, Y., 2015).Moreover, Gesture Search (Li, Y., 2010) allows users to use alphabet gesture combinations to filter results returned for the initial gesture.Although such gesture-based approaches have provided better mapping mechanism and more convenient access, there are still two issues.First, the mapping only supports a limited number of mappings as it largely relies on the text description of apps (i.e., 26 alphabets can be only mapped to 26 apps respectively).Second, they only work when a user knows the app's name or other relevant textual description in advance.That is to say, for locating apps users rarely use or never used after first installation, the search will become much less effective as users may not be able to recall any textual cues about them.

Icon style classification
Icon classifications are used to understand how icons can affect users' recognition and preference of those apps.Several classifications have been proposed recently.Chen (2015) groups app icons based on stylization (pictorial illustration, graphic rendering and graphic symbology) and border shape (open border and close border).Jakob (2014) divides icons into two groups based on the mapping between icons and what they actually represent: resemblance (depicting a physical object), reference (depicting object by reference or analogy), and arbitrary (have meaning by convention) groups.In this paper, a simplified classification with three groups (branded, described and combined) proposed by Jajszczyk ( 2015) is considered (see Figure 1).

Gesture shape classification
Touch gestures have become the dominant interaction technique when users interact with their touchscreen devices (e.g., tap, pinch, swipe etc.).Since every pattern a user can draw on the touchscreen by their fingers can be a potential gesture, it is also important to understand the main gesture shapes that users often draw as means of communication.In my research, Poppinga et al., (2014)'s classification is followed as they further divided metaphor shapes (Wobbrock et al., 2009) into five categories as shown in Figure 2.These categories are: abstract, geometric, drawing, letter and word shape.

User-defined gestures
Current gesture-based interaction method is overload and needs to expand the input vocabulary (McGrath, W. and Li, Y, 2014).Allowing enrolled user in the define process would be useful to enlarge the gestures' vocabulary (Wobbrock, J.O.,2009), because every touch trace that drawing by a finger can be a potential gesture.Moreover, Oh, U. and Findlater, L. ( 2013) have confirmed that user-defined gesture is efficient for user in the real word as expect.The vocabulary of user-defined gesture set is expanded by allow drawing shape gesture as input (Ouyang, T. and Li, Y., 2012).Additionally, user-defined gesture can receive a better recall performance than system defined ones (Oh, U. and Findlater, L., 2013) Evidence is also provided in (Nacenta, M.A, et al., 2013), which focused on the memorability of the gestures.Therefor in my study, we engaged end-user in the surface gesture define process to improve final gesture set's quality and users' satisfaction.
On the other side, in order to completely understand the preferences and habits of users in using gesture, the best way is to investigate from observation when users accomplished their task via gestures.In our research, we want to know what kind of gesture can help to raise the impression of apps which users are not familiar with, we will achieved by observation, and research gestures when users completing tasks such as defining and searching apps.

APPROACH
A user study collected both objective and subjective data was conducted.Objective data are images of touch gestures collected via Agile Gesture installed on experiment device, subjective data was collected at the end of the research via questionnaires.Target apps were pre-installed on the experiment devices.We first install the most popular apps according to the researcher's experience, and then after the participant come to the lab and ready for the experiment, researcher check participants' apps list and install what had missed on our experiment device.
In this research we divide the target apps into two groups: frequently used group and infrequently used group.During the data collection procedures we did not collect any information that may be harm to the participants, and we did not record the participant's name we just use a sequence number for reference.
A demonstration session was provided to help participants learn how to use Agile Gesture, followed by a self-practice session up to 15 minutes.After that participants were asked to complete 9 gesture creation tasks and answer the questionnaire pre-designed.

INITIAL RESULTS
Twenty four student volunteers, nine females and fifteen males, participated in the experiment.They are aged between 20 and 30 and studying different subjects.All of them are active smart-phone users, owning and using a smartphone for over 3 years.Table 1 shows the overview of the two groups (frequent and infrequent) with the total apps chosen by each group.In total, 216 gestures were created by these participants.The distribution of gesture shapes is shown in Figure 3.As seen in Figure 3, out of 216 gestures created by the participants, 36.1% have letter like shape, followed by 26.4% with abstract shape, 17.6% with drawing shape, 11.1% with geometric shape and 8.8% with word like shape.This confirms to the findings from the previous study (Poppinga et al., 2014) focusing on system functions, that users do not always create gestures of the same shape to access apps.Furthermore, our experiment has found that the shapes created for an app are closely related to two factors: (1) how frequently the app is used (see figure 4) (2) visual design of the app's icon (see figure 5).Participants tended to draw a letter like shape for frequently used apps while they tended to draw an abstract shape or drawing shape for infrequently used apps, as shown in Figure 4.This finding is in line with the participants' feedback provided in the questionnaire.The feedback from both the frequent and infrequent groups is illustrated in Figure 6.It can be clearly seen that the majority of the gestures (45.2%) created by the frequent group were based on app's name.For the infrequent group, 59.3% of the gestures were influenced by the functionality of an app.As a result, more drawing shape gestures were created than other shapes.This is probably due to a fact that the function factor often inspires user to go for a drawing shape gesture because the function is easy to present in the way of drawing.An app's icon also plays a role on user's creation of gestures, especially when the app is frequently used.If the app has a branded style icon, users tend to create a gesture based on the app's name.
For those apps adopting the descriptive icon design, the gesture is more likely drawn based on the app's function.This suggests that the type of an app's icon be taken into account while designing the builtin shortcut gestures.

MAIN CONTRIBUTIONS
Agile Gesture was presented to simulate a fast access tool using touch gesture shortcuts, with the assist of Agile Gesture we conducted a series of experiment to investigate the complex inner link and relationship between gesture shapes and app icon's style and app related impact factors respectively.A detailed understanding of users' mind model during the shortcut gesture setting process was provided.How these findings can influence in developing an informative shortcut gestures through visual cues presented on an app icon is introduced.The research contributed new understanding on user self-defined gesture shortcut.While, studies on understanding users' recall rate on self-defined gestures are still needed.The recall correctness of the gesture shape as well as the mapping, and the gesture recall rate after a period of time of creating need to be further investigated.

Figure 1 :
Figure 1: Examples of app icons' style

Figure 3 :
Figure 3: Overall distribution of gesture shapes

Figure 4 :
Figure 4: Group breakdowns of gesture shape distribution

Figure 5 :
Figure 5: Distribution of affect factors grouped by icon types

Figure 6 :
Figure 6 : Distribution of affect factors grouped by usage frequency of the app by

Table 1 :
Results overview McGrath, W. and Li, Y., 2014, October.Detecting tapping motion on the side of mobile devices by probabilistically combining hand postures.InProceedings of the 27th annual ACM symposium on User interface software and technology (pp.215-219).ACM.Morris, M.R., Wobbrock, J.O. and Wilson, A.D., 2010, May.Understanding users' preferences for surface gestures.In Proceedings of graphics interface 2010 (pp.261-268).Canadian Information Processing Society.