Opinion Mining, Sentiment Analysis, and Opinion Spam Detection

Feature-Based Opinion Mining and Summarization
(or Aspect-Based Sentiment Analysis and Summarization)
Detecting Fake Reviews
Opinion Lexicon ---- Datasets for Download ---- Talks ---- Publications
(Media coverage: The New York Times, The Economist, BusinessWeek and more ... )

New Book: Sentiment Analysis and Opinion Mining (Introduction and Survey), Morgan & Claypool Publishers, May 2012.

Textbook: Web Data Mining - Exploring Hyperlinks, Contents and Usage Data, Chapter 11: Opinion Mining, Second Edition, July 2011.

See "Feature-Based Opinion Mining and Summmarization" in Microsoft Live/Bing Search and Google Product Search (paper).

• Note: I don't know the techniques used by Microsoft Live/Bing (9/28/2007), but Google has a paper. To see the model, please check out (Hu and Liu, KDD-2004) and (Liu et al, WWW-2005) below, or the book above (better). Try search for a camera and click on reviews. You will see summarized user opinions on product features/aspects in a bar chart.

NLP Handbook Chapter: Sentiment Analysis and Subjectivity, 2nd Edition, (Editors) N. Indurkhya and F. J. Damerau), 2010.

Opinion Parser: A practical sentiment analysis system. It has used in the startup company, OpinionEQ.

• Analyze sentiments/opinions and emotions, extract entities, topics and their aspects/features/attributes, and handle sentiments in comparative sentences. The system is completely automatic.

New Tutorial: Sentiment Analysis Tutorial - (references), given at AAAI-2011, August 8, 2011 - (Check out the new book)

Recent Invited Talks and Interviews (Older Talks)

• From Opinion Mining to Modeling of Comments and Debates. Keynote talk at Conference for Big Data Systems, Applications and Privacy, March 10-11, 2013, Abu Dhabi.
• Modeling of Comments and Debates. Invited talk at IKDD workshop (Inaugural Workshop of India KDD Chapter), Feb 15, 2013, Mysore, India.
• Sentiment Analysis and Social Media Modeling. Invited talk at TCS Symposium, Feb 12, 2013, New Delhi, India.
• Sentiment Analysis and Social Media Modeling. Invited talk at the Marketing Department, Kellogg School of Management, Northwestern University, January 23, 2013.
• Analysis and Modeling of Opinions and Sentiments. Invited talk at Toyota Technological Institute at Chicago (TTI). Dec 4, 2012, Chicago, USA.
• Next Generation of Sentiment Analysis. Keynote talk at Visual Text Analytics 2012, Oct 15, 2012, Seattle, USA.
• Detecting Fake Reviews. Invited talk at Microsoft Research, Oct 16, 2012, Redmond, WA.
• Detecting Fake Opinions in Social Media. Keynote talk at KDD-2012 Workshop on Issues of Sentiment Discovery and Opinion Mining, Aug 12, 2012, Beijing, China.
• Modeling Opinions and Beyond in Social Media. KDD Summer School (Aug 10, 2012) at the KDD conference (Aug 12-16, 2012), Beijing, China. (This is an 2-hour lecture. I will talk about how to model opinions, comments, debates, etc. in social media).
• Analysis and Modeling of Sentiments and Opinions. Invited talk given at IBM Almedan Research Center, June 21, 2012.
• An interview: The Science of Detecting Fake Reviews. Interview by Trinity Hartman at Content Ping, May 18, 2012.
• Practical Sentiment Analysis. Invited tutorial at Sentiment Analysis Symposium, May 8 (tutorial on May 7), 2012, New York, USA.
• An interview: Practical Sentiment Analysis and Lies. Interviewed by Tom H. C. Anderson for his Next Gen Market Research blog, April 9, 2012.
• Older Talks

1. Introduction

This work is in the area of sentiment analysis and opinion mining from social media, e.g., reviews, forum discussions, and blogs. In our KDD-2004 paper, we proposed the Feature-Based Opinion Mining model, which is now also called Aspect-Based Opinion Mining (as the term feature here can confuse with the term feature used in machine learning). The output of such opinion mining is a feature-based opinion summary or aspect-based opinion summary. The commonly known sentiment classification is a sub-task. Our current work is in two main areas, which reflect two kinds of opinions (or evaluations)

• Mining regular (or direct) opinions. Ex: (1). This camera is great. (2). After taking the drug, I got stomach pain.
• Mining comparative opinions. Ex: Coke tastes better than Pepsi.

Since 2006, we have also worked on

• Fake review and opinion spam detection. Fake reviews are also called bogus reviews or fraudulent reviews. See the papers [WWW-2007, WSDM-2008, CIKM-2010a, CIKM-2010b, WWW-2012]

2. Sentiment Analysis or Mining of Regular Opinions

In this research, we aim to mine and to summarize online opinions in reviews, tweets, blogs, forum discussions, etc. Specifically, we mine features or aspects of entities (e.g., products) or topics on which people have expressed their opinions and determine whether the opinions are positive or negative. For opinion summarization, we advocate the quantitative aspect and the target of opinions because 50% of the people say something is bad is not the same as 5% say it is bad.

Abstraction of the problem: Feature-based opinion mining and summarization (aspect-based opinion mining and summarization) of multiple reviews (KDD-04 and WWW-05)
Formal definitions can be found in my book "Sentiment Analysis and Opinion Mining". They are based on several of our papers in 2004 and 2005. The abstraction provides a model of online opinions, describes what should be extracted from opinion sources (e.g., reviews, forums, and blogs) and how the results may be organized and presented to the user. The main mining tasks are:

• mining entities and their features (or aspects) that have been commented on or evaluated by people,
• determining whether the comment/opinion on each entity feature (or aspect) is positive, negative or neutral (aspect-based sentiment classification), and
• summarizing the results.

Opinion Lexicon (or Sentiment Lexicon)

• A list of positive and negative opinion words or sentiment words for English (around 6800 words). This list was compiled over many years starting from our first paper (Hu and Liu, KDD-2004).
• Although necessary, having an opinion lexicon is far from sufficient for accurate sentiment analysis. See this paper: Sentiment Analysis and Subjectivity or the Sentiment Analysis book.
• Try Search for the Best Restaurant based on specific aspects, e.g., "best burger," "friendliest service". The system uses the lexicon and is based on restaurant reviews.

Data Sets

• Customer Review Datasets (5 products) associated with the paper (Hu and Liu, KDD-2004).
• Additional Review Datasets (9 products) some used in (Ding, Liu and Yu, WSDM-2008), which improves the lexicon-based method proposed in (Hu and Liu, KDD-2004)
• Amazon Product Review Data (more than 5.8 million reviews) used in (Jindal and Liu, WWW-2007, WSDM-2008; Lim et al, CIKM-2010; Jindal, Liu and Lim, CIKM-2010; Mukherjee et al. WWW-2011; Mukherjee, Liu and Glance, WWW-2012) for opinion spam (fake review) detection. You can also use it for sentiment analysis. It has information about reviewers, review texts, ratings, product info, etc. Due to the large file size, you may need to use Download Accelerator Plus (DAP) to download. If you use this data, please cite (Jindal and Liu, WSDM-2008).
• Pros and cons dataset used in (Ganapathibhotla and Liu, Coling-2008) for determining context (aspect) dependent sentiment words, which are then applied to sentiment analysis of comparative sentiences (comparative sentence dataset). The same form of Pros and Cons data was also used in (Liu, Hu and Cheng, WWW-2005).
• Comparative sentence dataset used in (Jindal and Liu, SIGIR-06) and (Jindal and Liu, AAAI-2006). Comparative sentence dataset used in (Ganapathibhotla and Liu, Coling-2008).
• Blog author gender classification data set associated with the paper (Mukherjee and Liu, EMNLP-2010)

Publications - (sentiment analysis)                Publications - (opinion spam or fake review detection)

1. Bing Liu. Sentiment Analysis and Opinion Mining. Morgan & Claypool Publishers, May 2012.

2. Arjun Mukherjee and Bing Liu. Modeling Review Comments. Proceedings of 50th Annual Meeting of Association for Computational Linguistics (ACL-2012), July 8-14, 2012, Jeju, Republic of Korea.

3. Arjun Mukherjee and Bing Liu. Aspect Extraction through Semi-Supervised Modeling. Proceedings of 50th Annual Meeting of Association for Computational Linguistics (ACL-2012), July 8-14, 2012, Jeju, Republic of Korea.

4. Arjun Mukherjee and Bing Liu. Mining Contentions from Discussions and Debates. to appear in Proceedings of SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2012), Aug. 12-16, 2012, Beijing, China.

5. Lei Zhang and Bing Liu. "Extracting Resource Terms for Sentiment Analysis," Proceedings of the 5th International Joint Conference on Natural Language Processing (IJCNLP-2011), November 8-13, 2011, Chiang Mai, Thailand.

6. Zhongwu Zhai, Bing Liu, Lei Zhang, Hua Xu, Peifa Jia. Identifying Evaluative Opinions in Online Discussions. Proceedings of AAAI-2011, San Francisco, USA, August 7-11, 2011.

7. Lei Zhang and Bing Liu. "Identifying Noun Product Features that Imply Opinions." ACL-2011 (short paper), Portland, Oregon, USA, June 19-24, 2011.

8. Guang Qiu, Bing Liu, Jiajun Bu and Chun Chen. "Opinion Word Expansion and Target Extraction through Double Propagation." Computational Linguistics, March 2011, Vol. 37, No. 1: 9.27.

9. Zhongwu Zhai, Bing Liu, Hua Xu, Peifa Jia. "Constrained LDA for Grouping Product Features in Opinion Mining." Proceedings of PAKDD-2011, Shenzhen, China, 2011. (Best Paper Award)

10. Lei Zhang and Bing Liu. "Entity Set Expansion in Opinion Documents." Proceedings of the ACM Conference on Hypertext and Hypermedia (HT-2011), Eindhoven, Netherlands, June 6-9, 2011.

11. Zhongwu Zhai, Bing Liu, Hua Xu and Peifa Jia. "Clustering Product Features for Opinion Mining." Proceedings of Fourth ACM International Conference on Web Search and Data Mining (WSDM-2011), Feb. 9-12, 2011, Hong Kong, China.

12. Arjun Mukherjee and Bing Liu. "Improving Gender Classification of Blog Authors." Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-10). Oct. 9-11, 2010, MIT, Massachusetts, USA.

13. Xiaowen Ding and Bing Liu. "Resolving Object and Attribute Coreference in Opinion Mining." Proceedings of the 23rd International Conference on Computational Linguistics (COLING-2010), August 23-27, Beijing, China.

14. Zhongwu Zhai, Bing Liu, Hua Xu and Peifa Jia. "Grouping Product Features Using Semi-Supervised Learning with Soft-Constraints" Proceedings of the 23rd International Conference on Computational Linguistics (COLING-2010), August 23-27, Beijing, China.

15. Lei Zhang and Bing Liu. "Extracting and Ranking Product Features in Opinion Documents." Proceedings of the 23rd International Conference on Computational Linguistics (COLING-2010), August 23-27, Beijing, China.

16. Bing Liu. "Sentiment Analysis: A Multifaceted Problem." Invited paper, IEEE Intelligent Systems, 25(3), 2010, pp. 76-80.

17. Bing Liu. "Sentiment Analysis and Subjectivity." Invited Chapter for the Handbook of Natural Language Processing, Second Edition. March, 2010.

18. Ramanathan Narayanan, Bing Liu and Alok Choudhary. "Sentiment Analysis of Conditional Sentences." Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-09). August 6-7, 2009. Singapore.

19. Guang Qiu, Bing Liu, Jiajun Bu and Chun Chen. "Expanding Domain Sentiment Lexicon through Double Propagation." Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI-09), Pasadena, California, USA, July 11-17, 2009.

20. Xiaowen Ding, Bing Liu and Lei Zhang. "Entity Discovery and Assignment for Opinion Mining Applications," Proceedings of ACM SIGKDD Interntaional Conference on Knowledge Disocvery and Data Mining (KDD-09, industrial track), June 28-July 1, 2009, Paris.

21. Bing Liu. "Opinion Mining." Invited contribution to Encyclopedia of Database Systems, 2008.

22. Murthy Ganapathibhotla and Bing Liu. "Mining Opinions in Comparative Sentences." Proceedings of the 22nd International Conference on Computational Linguistics (Coling-2008), Manchester, 18-22 August, 2008.

23. Xiaowen Ding, Bing Liu and Philip S. Yu. "A Holistic Lexicon-Based Appraoch to Opinion Mining." Proceedings of First ACM International Conference on Web Search and Data Mining (WSDM-2008), Feb 11-12, 2008, Stanford University, Stanford, California, USA.

24. Xiaowen Ding and Bing Liu. "The Utility of Linguistic Rules in Opinion Mining." SIGIR-2007 (poster paper), 23-27 July 2007, Amsterdam.

25. Nitin Jindal and Bing Liu. "Identifying Comparative Sentences in Text Documents" Proceedings of the 29th Annual International ACM SIGIR Conference on Research & Development on Information Retrieval (SIGIR-06), Seattle 2006.

26. Nitin Jindal and Bing Liu. "Mining Comprative Sentences and Relations." Proceedings of 21st National Conference on Artificial Intellgience (AAAI-2006), July 16.20, 2006, Boston, Massachusetts, USA.

27. Bing Liu, Minqing Hu and Junsheng Cheng. "Opinion Observer: Analyzing and Comparing Opinions on the Web"Proceedings of the 14th international World Wide Web conference (WWW-2005), May 10-14, 2005, in Chiba, Japan.

28. Minqing Hu and Bing Liu. "Mining Opinion Features in Customer Reviews." Proceedings of Nineteeth National Conference on Artificial Intellgience (AAAI-2004), San Jose, USA, July 2004.

29. Minqing Hu and Bing Liu. "Mining and summarizing customer reviews." Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-2004, full paper), Seattle, Washington, USA, Aug 22-25, 2004.

Publications - (Opinion spam or fake review detection)           (Check out my Opinion Spam Detection project homepage)

1. Arjun Mukherjee, Bing Liu, and Natalie Glance. Spotting Fake Reviewer Groups in Consumer Reviews. International World Wide Web Conference (WWW-2012), Lyon, France, April 16-20, 2012.

2. Guan Wang, Sihong Xie, Bing Liu, Philip S. Yu. Identify Online Store Review Spammers via Social Review Graph. ACM Transactions on Intelligent Systems and Technology, accepted for publication, 2011.

3. Guan Wang, Sihong Xie, Bing Liu, Philip S. Yu. Review Graph based Online Store Review Spammer Detection. ICDM-2011, 2011.

4. Arjun Mukherjee, Bing Liu, Junhui Wang, Natalie Glance, Nitin Jindal. Detecting Group Review Spam. WWW-2011 poster paper, 2011.

5. Ee-Peng Lim, Viet-An Nguyen, Nitin Jindal, Bing Liu and Hady Lauw. "Detecting Product Review Spammers using Rating Behaviors." The 19th ACM International Conference on Information and Knowledge Management (CIKM-2010, full paper), Toronto, Canada, Oct 26 - 30, 2010.

6. Nitin Jindal, Bing Liu and Ee-Peng Lim. "Finding Unusual Review Patterns Using Unexpected Rules." The 19th ACM International Conference on Information and Knowledge Management (CIKM-2010, short paper), Toronto, Canada, Oct 26 - 30, 2010.

7. Nitin Jindal and Bing Liu. "Opinion Spam and Analysis." Proceedings of First ACM International Conference on Web Search and Data Mining (WSDM-2008), Feb 11-12, 2008, Stanford University, Stanford, California, USA.

8. Nitin Jindal and Bing Liu. "Review Spam Detection." Proceedings of WWW-2007 (poster paper), May 8-12, Banff, Canada.

Created on May 15, 2004 by Bing Liu; and Minqing Hu.