Intelligent media indexing and television recommender systems

This paper presents a state-of-art review on recommender systems that has been identified as possible areas of my research. It describes the current generation of collaborative filtering methods which are usually classified into three main categories: item-based, user-based and hybrid methods. The paper considers these methods to be applied to digital television providing recommendation for viewers. Personalised television is predicted to be the next step in the evolution of television which might reshape the whole landscape of mass media. The paper also identifies anticipated problems in the domain of recommender systems which includes indexing, collaborative filtering, ranking problems and possible research directions to solve these problems. Finally, these challenges are considered in the domain of personalised television which has its own inherent shortcomings.


MOTIVATION FOR RESEARCH
The digital switchover will be completed by 2012 in the UK it implies that all households will have access to some kind of digital television content.Digital television offers a wider range of channels where the user can access a large amount of content thought terrestrial, satellite or cable broadcast.Video on demand (VOD) and electronic program guide (EPG) services are part of this expansion, which help to make digital television more flexible and provide easy to access information.
In economy Anderson (2008) introduced the concept of long tail distribution, it shows that retailers sell relatively large quantities from small number of popular items and sell large number of items which are not that popular in smaller quantities.This distribution can be applied to multimedia item, since multimedia items can also be considered as products.This suggests that there is a direct correlation between users' interest and the popularity of multimedia items (FIGURE 1), that means popular items meet only a small part of the populations' taste.The reason why popular items get popular despite that they are only meet the interest of a small proportion of the population can be explained by the nature of mass media, people tend to watch programs that are available in peak time and usually heavily advertised, which they would not normally watch.Therefore, mass media pulls together people who would not normally find themselves together (Anderson 2008).However, VOD services enable non linear access to content, so it is possible to serve viewers who have less popular taste, which cannot be satisfied by conventional television.This would reduce the popularity of currently popular items and create a flatter distribution.However, the variety of content would inevitable create information load so users might find it difficult to find programs that they wish to watch, and sometime they spend more time browsing channels or browsing the digital library than watching actual programs.In addition to that users might not know what exactly they are interested in, therefore it is not possible to them to search explicitly for content.To overcome this personalisation systems aim to close the gap between potential interests and available items by suggesting items that match to that interest.However, providing reliable recommendation gets more problematic as the system deals with the 'long tail' of the data.
Personalisation is a powerful technique which might reshape the whole landscape of mass media, personalised television is predicted to be the next level in the evolution of interactive television.Personal TV suggestion engines (Wang et al. 2008) have appeared in recent years, offering personalised TV listings, personalised content channels (e.g.beeTV), etc.However, these services only adopted a simplified version of online recommendation system, because of the inherent problems with digital television.They usually offer personalised services based on genre or program types (e.g.thriller, comedy) manually set by the user, only very few of them takes into account previous user preferences.
Since the technology enables service providers to keep a log of user activity, this information can be used as a basis for research to develop a retrieval tool which would provide personalised recommendation.In order to provide efficient recommendation this project also aims to develop an indexing tool that will create sufficient data for retrieval.
The proposed system would contain five main modules (FIGURE 2).A user profile module would be in charge of monitoring and logging users' behaviour and storing this data in the database.It would also modify user data based on pieces of information on how users perceive items that are presented to them (profiling).The other information gathering module would create metadata for multimedia content.This might include a collaborative tagging system and an automatic metadata retrieval tool.
The main part of the system is the recommendation engine.This module would return a list of recommended items for any given user using the two datasets (users, items).It also receives feedback from users that will directly affect the procedure.This list will be passed to the post-filtering module which would filter recommended items based on contextual information.The last part of the process would rank the results by popularity, date, etc. and present it to the user.The rest of the paper will present a brief overview of the work that is relevant to this project in indexing and multimedia retrieval.The second part will focus on the anticipated research problems in general and in the television recommendation system domain.

BACKGROUND AND RELATED WORK
Recommender systems are a means of personalisation providing users with personalised recommendation that would possibly suit users needs.Recommender systems are used in a broad range of applications and web services.

Collaborative filtering
One of the most popular techniques that are used for recommender systems is collaborative filtering (Herlocker et al. 1999).It enables the system to build coherent communities based on shared taste and behaviour.These feedback profiling methods use the opinion of users to help individuals to identify content of interest from a large set of choices, which otherwise would be problematic to find.The basic idea is that recommendations are provided on the basis of user profiles and on the metadata of particular items.The result is a set of recommended items that have not been known to the user before, but match to his or her taste.These techniques are based on two basic assumptions, first of all, users who like the same documents are assumed to have the same interest.Secondly, it is assumed that users taste is relatively constant.
User-based filtering techniques (Wang et al. 2006) concentrate on finding similar users to the active user.First a set of nearest neighbours of the target user are computed.This is performed by computing correlations or similarities between user records and the active user.Then, different methods are used to combine the neighbours' item ratings to produce a prediction value for the target user on unrated items.The major problem with this approach is the bottleneck problem, the complexity of the system increases as the number of users grows which could reach an unmanageable number of connections to compute in large commercial systems.
Item-based collaborative filtering (Sarwar et al. 2001) aims to find items which are similar to a particular user's preferences.This algorithm attempts to find similar items that are co-rated (or visited) by different users similarly.This is done by performing similarity calculation between items.Thus, item-based algorithms avoid the bottleneck in user-user computations by first considering the relationships among items.
It has been shown (Adomavicius and Tuzhilin 2005) that hybrid recommender systems are more efficient than systems based purely on item-based or user-based collaborative filtering.In addition to that data sparsity is considered one of the main reasons why these systems perform poorly.Wang et al. ( 2006) proposed a hybrid system where user-based and item-based collaborative filtering approaches are unified using probabilistic fusion framework, it is showed that this system performs better even with sparse data.

Indexing
In order to build a useful database for retrieval, pieces of information about multimedia items are retrieved from the EPG provider.In some systems (Tsunoda and Hoshino 2008) automatic metadata expansion is used to convert this information to metadata.However, systems which are aimed to unite content from different providers might find it problematic to obtain the same set of information for each multimedia item, since different providers have different internal data structures.
Alternatively, collaborative tagging processes offer techniques to add some extra information in order to enrich the metadata of a multimedia item.Concerns were raised on the reliability of this method, since users might assign tags in an uncontrollable manner, which would result unsystematic metadata.However, some researchers (Golder and Huberman 2005) found stable patterns in the structure of these tags.It is suggested that the proposition of tags becomes fixed if enough users assigned tags to a particular item.After a threshold is reached, 'each tag's frequency is a nearly fixed proportion of the total frequency of all tags used' (Golder and Huberman 2005, p. 6).

Lack of semantic descriptions
Since the traditional way of multimedia retrieval can be problematic, because of the gap between low-level features of the multimedia items and the semantic symbols that is used for retrieval.Recently researchers begin to explore social multimedia signals (Chang 2008) that can be fused with traditional features of content-based multimedia retrieval which might result more effective multimedia retrieval.Interaction of users can be used to identify low level features, for example adjectives that are used in a descriptive manner might refer to some of the features of the multimedia item.

User interests' predictions
Collaborative filtering is a very well research area, because it can be applied to a wide range of problems, for example predicting user's taste in retail, music, cinema etc.It can also offer prediction to estimate if a particular product would be successful or not.Therefore providing efficient recommendation or prediction is a key to success for commercial applications.
One of the main problems that might arise in online recommendation systems is the so-called cold start problem (Schein et al. 2002).Until a certain point, there is not enough data to make sufficient recommendation, in order to overcome this, systems usually set a limit (e.g.Netflix) which has to be reached in order to provide enough data for the recommendation engine.On the item level this problem appears differently.Since old items have more rating than new ones, recommender systems tend to be biased towards the old and have difficulty showing the new.This might be the problem, since users tend to prefer new items over old ones.One of suggestions to solve this problem (Lee et al. 2009) is to take temporal information into account, temporal information includes the launch time of the item and the time difference between the item was rated and the launch time.
One of the assumptions discussed above was that users' taste does not change significantly over time (Boyer and Brun 2007), however, some recommender systems consider users' taste within a shorter time frame, and assume that in a longer term users' profiles can expire (Tsunoda and Hoshino 2008).In addition to that a system can separate a core taste of the user and a temporary taste which can be computed by comparing similar items in users' profiles.Then the core taste could be used in a longer period of time while the temporary taste can expire if the user becomes interested in items that do not match her previous temporary taste.
Popular movies and unpopular movies has a shape of distribution shown in FIGURE 3 which make the movie easy to predict, because the majority of users grouped together either on the positive side or the negative side of the scale.In FIGURE 4 both movies have the same average rating (5/10), however, the distribution is very different.Items which people either love or hate are more problematic to predict.This can be accounted to fact that these movies have only good and bad ratings and nothing in between that makes it problematic to match it to users' profiles.For example users who loved or hated Michael Moore hates America clearly based their rating on political orientation rather than previous taste so from the point of known recommender algorithms users behave unpredictably.Also, unpredictable items might not linked very closely to the rest of the system, because they represent a new style or represent a theme that is unique, which also makes it difficult to predict whether it will be liked or not.Current recommender system do not differentiate between item and item in their internal data structure, thus these differences cannot be taken into account during the process of recommendation.Therefore, we aim to investigate this matter further by indentifying items that should be considered differently based on the distribution of the votes or other criteria (e.g.other systems found them unpredictable) and developing strategies that can be tailored to threat a particular item individually and use strategies that fit best for that item to provide efficient recommendation.
Since previous votes tend to influence future votes and users' taste changes over time, it is possible that the shape of the distribution changes as the item receives more votes.FIGURE 5 illustrates the idea, each column represents the vote distribution after a particular number of votes.It shows that the semantic orientation of the vote distribution can change significantly over time as movies receive a high volume of votes that differ from the current distribution.This measure would represent the fluctuation in users' taste over time.In this case (FIGURE 5) users tend give rating four and five more to the movie Silkwood compared to the initial distribution (after 500 votes).That suggests that users like the movie more than they initially liked it.In this case it is a shift towards higher votes, but there could also be a two directional shift, that is a shift from the middle towards both ends of the scale.

Contextual information and ranking problems
A recommender engine would return many items that are potentially interesting to the user, for that reason it is suggested to apply a post-filtering algorithm which would return only a few items that can be displayed on screen.Also, recommender systems consider user profiles as a homogeneous system and take into account items equally.However, in reality taste is more a multidimensional system which might change with time, influenced by other users etc.For example a user might prefer to look different items at the weekend than at weekdays or his or her taste might change slightly after they watched a particular movie.These subtle modifications of the system could be achieved by post-filtering results returned by the recommender engine.Contextual information (e.g.user's age, favourite genre) (Adomavicius et al. 2005) can be used to filter results returned by the recommendation engine.Since these parameters can change quicker than the general taste of the user, contextual information should be considered separately, not as part of the recommender system.These techniques are useful to differentiate the core taste and the temporary taste of the user to threat time as a separate dimension.
General sentiment score is an indication of the general popularity of a multimedia item that can be obtained from different sources using NLP (Natural Language Processing) techniques.For example the frequency of particular adjectives that occur near to a multimedia item (e.g. in comments, in reviews) can give an indication of how well that item is perceived by the general public.This piece of information can be used to rank the search results and give the user the most popular items first within the domain of their preference.This information would provide a different way of measuring data compared to ratings, since both measurements express sentiment towards an item in different ways.However, it is very likely that these measurements do not differ significantly, since they only represent different ways of expressing opinion.

Program clustering on digital television
In addition to problems anticipated in online recommender system, recommender systems of interactive television suffer from several inherent problems.It includes primitive user interface, poorer access to information, the services providers' monopoly on supplying software for the digital box (as opposed to a more comparative environment on the web).
Since a television set might be shared between people who are living in a same household, it is problematic to determine who is watching a program at any given time.One solution might be to set up a user profile system, which would enable users to log into their TV sets.From that point the system would function as a standard recommendation system.The other solution would be to take contextual information into consideration, for example timestamp could be used to determine who is watching the television at any given time based on the assumption that similar type of programs are likely to be watched on the same television set at the same time of each day (e.g.news in the morning).Based on previous viewing history, clustering techniques can be used to build a 'family profile' to differentiate potential program sets that might be watched on a particular television set.After different sets have been identified a probability score can be assigned to all sets that would be used to predict which profile would be in use at any given time for recommendation.This solution has a drawback that efficient feedback system cannot be integrated into the system since users cannot be identified.It is also possible to create a hybrid system that would use different strategies to offer recommendation depending on whether the user is logged in or not.

Feedback and evaluation
The efficiency of the system can be evaluated by direct feedback from users.Users can rate whether the recommended programs are interesting to them.Alternatively, users can provide implicit feedback by simply watching recommended movies (Yehuda 2008).This is based on an assumption that if the user watches a multimedia content to some length, that item is interesting to them.In the digital television domain, feedback and evaluation of the system would depend how users are identified, if the program clustering technique is used to predicted who is watching the television set, it is more problematic to implement a reliable feedback system, since the feedback would be based on the same prediction that determines who is watching the program, it is questionable whether that prediction could be used to determine who gives the feedback.

CONCLUSION
Recommender systems made significant progress over the years.However, the current generation of recommender systems still require further improvements.Since, recommender systems applied to television still in its infancy, there are many challenges that researchers face.This paper identified the main areas of my research in the domain of personalised television.It also considered possible solutions to the anticipated problems.It is important to note that these problems are defined vaguely only considering the bigger picture.Therefore, future work should be done to understand the problem space in order to provide efficient solutions to the identified problems.

FIGURE 1 :
FIGURE 1: User's interest and the popularity of content (illustration)

FIGURE 2 :
FIGURE 2: Social information retrieval that combines user intelligence with the content features