2014年10月30日 星期四

Recommender systems

We had learned the concept of recommender systems last week and it’s so interesting and useful.
As the development of social media and IT technology, data and information continued to increase, and it is a big problem for users. How to get the exact data needed by user is very important.
Recommender systems assist and augment this natural social process. In a typical recommender system people provide recommendations as inputs, which the system then aggregates and directs to appropriate recipients. In some cases the primary transformation is in the aggregation; in others the system’s value lies in its ability to make good matches between the recommenders and those seeking recommendations.
Personally speaking, I think the collaborative filtering is very interesting. developers of the first recommender system, Tapestry[1] , coined the phrase “collaborative filtering” and several others have adopted it. We prefer the more general term “recommender system"for two reasons. First, recommenders may not explicitly collaborate with recipients, who may be unknown to each other. Second, recommendations may suggest particularly interesting items, in addition to indicating those that should be filtered out. And it can be divided into memory-based and model-based.
For example, Jingdong is one of the biggest online shopping website in China. When I go for some interesting stuffs like "iPhone6". And when I decide to buy this, I can see the recommender system give me some advices.

But Recommender systems introduce two interesting incentive problems. First, once one has established a profile of interests, it is easy to free ride by consuming evaluations provided by others. Moreover, as Avery and Zeckhauser argue, this problem is not entirely solved even if evaluations are gathered implicitly from exist-ing resources or from monitoring user behavior.

Reference:
[1]1.Goldberg, D. Nichols, D., Oki, B. M., and Terry, D. Using collabora-tive filtering to weave an information tapestry. Commun. ACM 35, 12 (Dec.1992), 61—70

2014年10月17日 星期五

Social Network Analysis (SNA)

In lecture 6, Rosanna mentioned SNA and Betweenness Centrality. These two wonderful concepts impressed me a lot. Social networks really matter. In the popular press, the term social network usually refers to online communities such as Facebook. The relations between actors that they maintain or that are imposed on them expand or limit their potentialities in acting. For instance, an actor’s social network influences its opportunities to communicate, to exchange goods or to ask somebody for help. An actor may be part of many networks that consist of friends, relatives, neighbors, teammates. We can use graph to analyze social network because graphs are mathematical structures used to represent relationships between objects.[1]

Graph:
Since a social network is conceived to consist of relations among a bounded set of actors, the typical data model is a graph G = (V, E) with vertices V representing actors, edges E representing relations, and potentially many attributes for both.
Graph can be divided into directed graph and undirected graph.


Figure 1. Directed Graph

Betweenness Centrality:
Betweenness centrality measures a node’s potential on a path between two other nodes. For that purpose for each pair among the remaining nodes all shortest paths the particular node lies on are counted. Each number is then related to the overall number of shortest paths between the particular pair including the ones that do not pass the node the measure is calculated for. These shares are then added up for all pairs. For standardization the result is divided by the highest possible number of shortest paths a node may lie on which is (n−1)(n−2)/2 = n2−3n+2/2 .[2]


Reference:
[1] Developing a Social Network Analysis and Visualization Module for Repast Models
[2] Wei Li, Tian Tang, Lei Li. Mining. Social Network Analysis and Visualization

2014年10月3日 星期五

Emoticons in social media

People increasingly depend on social media to communicate and share information with others nowadays. Besides text, they frequently use emoticons to express their feeling and emotion, such as ^_^.

The rapid growth of social data from such online social networks as Twitter and sina weibo has aroused enormous interest in the mining and discovery of customer insights. But are the emoticons kinds of true reflection of people’s real emotion?

We have to review the concept of sentiment classification which has been mentioned by Rosanna in last week. A document or sentence can be classified by using dictionary-based method. Each word is given a score, along with three different dimensions: positive, negative and objective. By using the same method, we can classify any specific emoticon.

According to the investigation, the top 10 most frequently used emoticons in the corpus are listed in Table 1[1]. It demonstrates that although the number of emoticons is big, only a handful of emoticons are used far more frequently than all others.

                                                     TABLE 1. TOP 10 EMOTICONS


After knowing the sentiment of a specific emoticon, we can analyzing and classify the text content generated by user in sentence level by using dictionary-based method. And we can compare the two scores of emoticon and text sentence to see whether the emoticon reflects the emotion of user correctly.


Reference:

[1] Martin Min, Tanya Lee, Margaret Zhang, Lei Li
Netbase Solutions, Inc. Mountain View, USA