A Gamification Framework for Enhancing Search Literacy

Recent studies reveal that the overall search literacy leaves something to be desired and that the most people overestimate their skills in the domain of Web search. In this paper, a gamification framework is introduced that aims at increasing the search literacy within a ludic environment to support Internet users in being more successful in their daily search sessions. Furthermore, the paper presents plans for future work that strive for answering whether the search literacy can be improved by applying the presented approach and how we can detect and especially measure such an improvement.


INTRODUCTION
As the size of information on the World Wide Web increases, the effective use of Web search engines for information retrieval (IR) becomes a key challenge for Internet users.Search engines act as gatekeepers and provide access to online resources.Internet users are facing the challenge of finding the desired information and need therefore skills that can be broadly summarized under the term "information literacy".A person is information literate if she is able to recognize when information is needed and has the ability to locate, evaluate, and use effectively the needed information.This paper focuses on a specific aspect of information literacythe search literacy.The latter concept refers directly to the retrieval process of information and denotes the abilities to locate and access desired information in order to satisfy information needs with efficiency and effectiveness.
A recent study revealed that most users have a rather limited search literacy as well as they overestimate their competence in this area (Stark et al. (2014)).Bateman et al. (2012) stated that most searchers do not know how to use search engines effectively for satisfying their information needs.One possible reason is that beyond query completion, search engines provide no feedback to users that would help them in improving their search behavior.White (2013) observed that users struggle with finding an answer to a yes-no question; moreover, biases arising from people's beliefs influence their judgment, decision making and actions.Kodagoda and Wong (2008) showed that low literacy users take significantly more time than high literacy users to complete an information task, and were significantly less accurate.In addition, low literacy participants spent more time on a Web page to find the desired information.
These findings show that there is a need to improve the search literacy of Web search engine users.In this paper, we describe an approach for enhancing search skills via gamification in the domain of Web search engines to help users being more successful in their searches.For this purpose, we have developed a gamification framework that features different kinds of tasks including search and educational tasks in terms of Web search engines.The main goal is the gamification of the core IR task, i.e., Web search.We aspire noticeable and sustainable improvements of skills in the context of Web-based IR.

Search literacy
Producing accurate search results requires knowledge about the basic functioning of search engines as well as the following aspects: searchability, linguistic functions, query language, and ranking (Fuhr (2014)).According to Fuhr, a search-literate user needs to know appropriate search tactics and strategies in order to succeed in satisfying information needs effectively.
Searchability Users need to be aware of unsearchable content, since not all online resources that can be accessed via browsers, are findable.For example, the language used in the search query, the document type, and the recency of Web pages are some contextual factors into account as they may determine 4 reasons why Web resources can not be found.

Linguistic functions
Search engines apply linguistic functions to search queries such as word normalization, lemmatization, and phrase identification, and take composites and synonyms into account to deal both with the vagueness and the ambiguity of natural languages which form a crucial IR problem.

Query language
The use of search operators (e.g., Boolean operators, number ranges, facets, and field and URL predicates) and search options (e.g., for restricting the time, place, language, and document type of result items) allows users to express complex information needs and leads to more specific search results.
Ranking Studies exposed that users prefer search results on the first result page; items below the fold are seldom clicked on (Ho ¨ chsto ¨ tter and Lewandowski (2009)).Hence, it is important to produce result sets where relevant documents are located at a top position on the first page and in the visible area without the need to scroll by formulating precise queries for not missing relevant information.
Tactics and strategies Expressing complex information needs usually requires a series of steps and search queries.Strategies are plans for performing a complex search whereas tactics denote single operations to advance searches.Bates (1979) distinguishes between the following types of tactics: monitoring, file structure, search formulation, and term.

Gamification
Gamification is defined as a process of integrating game mechanics into non-game contexts to invoke gameful experiences and to engage users in solving problems (Zichermann and Cunningham (2011)).On the other side, if applied improperly, gamification can reduce the internal motivation or even replace it by external motivation (Nicholson (2012)).The concept is a viable means for increasing users' motivation, shaping users' behavior, and enhancing online services with (motivational) affordances to invoke ludic experiences.It primarily takes advantage of the fact that games are fun (Poels et al. ( 2007)).When developing gamification strategies, designers have to be aware of different kinds of gamers since each player has different motivations, in-game behaviors, and play styles (Dixon (2011)).Although there is still a lack of empirical evidence on the side effects of employable game elements, the findings of various studies (e.g., Hakulinen et al. (2013); O'Donovan et al. ( 2013)) lead to the conclusion that gamification does not harm the internal motivation of users at all if applied in a user-centered fashion (Nicholson (2012)).On the contrary, points, levels, leaderboards, and badges are an easy and effective way to increase users' performance.Nevertheless, one should take social and whether the employed game elements diminish (see Mekler et al. (2013)) or even suppress internal motivation.

FRAMEWORK
Traditional training methods and tutorials are meant to be a potential means to enhance the abilities of people.We focus on gamification where users have to master problems in a playful manner.Our approach aims at increasing users' Web search skills and basic understanding of the functioning of Web search engines.By "playing our game", users are to learn and to develop sustainable Web search strategies for satisfying their information needs subconsciously.In the best case, people will be aware of their new skills.We expect users noticing a (significant) improvement of their Web search skills as well as the precision of the search results.

Architecture
Our Web framework consists of a client-side (front end) and server-side (back end) component where appropriate handlers are capable of communicating with each other to handle user input and to change the application state as a result.Front end handlers validate and forward user input, received from user interfaces, via messages to the corresponding back end handlers that are interconnected with other modules.We put emphasis on a module based architecture that allows for an easy extension by features, tasks, and game design elements.In this context, a database serves for storing task-, user-and applicationrelated data.The front end is accessible via modern browsers and features a high usability (see section 4) that allows users to achieve the specified goals with effectiveness, efficiency and satisfaction.
The core of our developed framework is a search proxy that is used to retrieve search results from Web search engines.Our main goal is the gamification of Web search where users have to solve interactive search tasks.Microsoft's Bing1 suits our needs, since it offers a wide range of features.We exploit this rich feature set to provide a complete search interface containing all commonly employed search functions to our users.Depending on the received search query and search options from back end handlers, the proxy creates a URL to access the Bing search API.Search results are received in textual form, i.e., in JSON format, and need therefore to be parsed first.Once appropriate objects have been created, the results are forwarded to the initial caller, the front end handler, which renders them along other information according to task related settings.
Basically, users are provided with different types of tasks they have to solve that allow the application of various strategies to enhance search literacy on different levels.
To emphasize the playful character of the application, we introduced the notion of "game mode"each game mode is responsible for one type of task.Being able to detect an improvement of search skills requires keeping track of a user's progress in the "game".In this context, the logging of user interactions and corresponding results contributes to the creation of user profiles in order to analyze and to disclose the "play behavior" of our users.

Quiz
This game mode aims at determining as well as increasing the basic knowledge of users related to Web search engines.Users are presented with questions they have to answer in terms of selection.Depending on the question type (single or multiple choice), one has to select preset answers one considers to be correct.
Figure 1: Additional information that contributes to the understanding of the answer can be accessed through the information icon next to each one.

Search hunt
The purpose of this game mode is to support users in increasing the precision of their search queries and in improving their relevance judgment and content-finding skills.In search hunt tasks, we ask questions for which users have to find the corresponding answers using a feature-rich search interface that is connected to the aforementioned search proxy.Answers are to be found by formulating search queries and identifying as well as exploring relevant search results.

Figure 2:
In this scenario, a user has to create a search query to find the desired image.The task interface offers hints and clues that users can request for a fee, i.e., points, as well as an option for skipping the current task without evaluation if desired.

Query tuning
As the name suggests, the game mode aims directly at increasing the precision of search queries.However, the methods differ from search hunts.A Web page (URL) and related details are supplied.Users have to create a search query that ranks the given Web page at a top position within the results list received by the search proxy.The closer to the first position the given Web page is, the better users are rewarded.While refining (tuning) search queries to solve the task, searchers are also to reflect on the changes occurring in the results where even small modifications to the query may lead to very different results.

Game design elements
The framework is provided with points, levels, badges, leaderboards, and sound effects.Points are received for solving tasks partially or fully correct.In addition, one can receive bonus points, e.g., for quick solving tasks.Levels specify a user's current game state and the task difficulty: the higher the level is, the more difficult are the tasks users are presented with.By exceeding point thresholds, one can reach the next levels.User interactions such as "solving a task" or "reaching x points in game mode y" are connected with badges.When triggered, the user is rewarded accordingly in case the appropriate conditions are met.Leaderboardsone per game mode allow simple comparisons, promote competition, and serve as a powerful motivator for continuing.We represent leaderboards as ordered lists with a point score beside each name.Along with levels, leaderboards indicate that players have status or achievement in the game.Sound effects are used to guide users and to introduce events, such as the receipt of points/badges, the beginning/ending of a task, and the reaching of the next level.

FIRST RESULTS
First of all, we measured the usability to test whether the user interface is usable.Concerning this matter, we created demonstration tasks for each game mode that have been solved during a user test by N = 15 participants.At the end of each lab session, the System Usability Scale (SUS) was used for measuring perceptions of usability.A total score of 90.2 allows us to conclude that the framework (or more precisely, the user interface) is userfriendly and capable of increasing users' motivation.

CONCLUSION AND OUTLOOK
We created a framework that features a high usability in order to fulfill the desired goal effectively and expect to improve the quality of search sessions which will lead in general to a higher task completion rate.The next step is to create a rich set of challenging tasks to train Web search engine users.Each game mode can be used to enhance different aspects of search literacy.For instance, quiz tasks provide more insights into the functioning of search engines.Search hunt tasks can be designed in a manner that the users have to explore various commonly employed search engine interface features, such as search options or operators, to find appropriate answers.As different result types provide different options, solutions can be located on Web results as well as on images, news, or even in videos.Query tuning tasks support users in creating precise search queries within a step-by-step refining process.
After the creation of our task set, we will run a longterm study with a large user base.The outcomes of this study will help us to tune our system and to generate ideal solutions for each task which we will use to score individual users by the closeness of their solution to the ideal one.Furthermore, we will identify key factors that make a user search literate and compare users' knowledge to the initial one to answer whether and under what circumstances the search literacy can actually be improved.In addition, we plan to invite once more interested participants after a certain period of time who will solve new tasks but with the same complexity in order to test the sustainability of the improvements.

Figure 3 :
Figure 3: Users have to enter a search query in a text field which will be forwarded via the search proxy to the connected search engine to rank the URL marked in green at a top position.The words marked in red are terms or search operators that must not appear in the search query.Users are free to finish a task at any time.The evaluation encompasses the last search process and the amount of needed attempts.