Opinion Mining, Sentiment Analysis, and Opinion Spam Detection

Feature-Based Opinion Mining and Summarization
(or Aspect-Based Sentiment Analysis and Summarization)
Detecting Fake Reviews
Opinion Lexicon ---- Datasets for Download ---- Talks ---- Publications
(Media coverage: The New York Times, The Economist, BusinessWeek and more ... )


New Book: Sentiment Analysis and Opinion Mining (Introduction and Survey), Morgan & Claypool, May 2012.

Textbook: Web Data Mining - Exploring Hyperlinks, Contents and Usage Data, Chap 11: Opinion Mining, 2nd Edition, July 2011.

See "Feature-Based Opinion Mining and Summmarization" in Microsoft Live/Bing Search and Google Product Search (paper).

NLP Handbook Chapter: Sentiment Analysis and Subjectivity, 2nd Edition, Eds: N. Indurkhya and F.J. Damerau, 2010.

Opinion Parser: the sentiment analysis system used in the startup company, OpinionEQ.

New Tutorial: Sentiment Analysis Tutorial - (references), given at AAAI-2011, August 8, 2011 - (Check out the new book)

Recent Invited Talks and Interviews (Older Talks)

1. Introduction

This work is in the area of sentiment analysis and opinion mining from social media, e.g., reviews, forum discussions, and blogs. In our KDD-2004 paper, we proposed the Feature-Based Opinion Mining model, which is now also called Aspect-Based Opinion Mining (as the term feature here can confuse with the term feature used in machine learning). The output of such opinion mining is a feature-based opinion summary or aspect-based opinion summary. The commonly known sentiment classification is a sub-task. Our current work is in two main areas, which reflect two kinds of opinions (or evaluations)

Since 2006, we have also worked on

2. Sentiment Analysis or Mining of Regular Opinions

In this research, we aim to mine and to summarize online opinions in reviews, tweets, blogs, forum discussions, etc. Specifically, we mine features or aspects of entities (e.g., products) or topics on which people have expressed their opinions and determine whether the opinions are positive or negative. For opinion summarization, we advocate the quantitative aspect and the target of opinions because 50% of the people say something is bad is not the same as 5% say it is bad.

Abstraction of the problem: Feature-based opinion mining and summarization (aspect-based opinion mining and summarization) of multiple reviews (KDD-04 and WWW-05)
Formal definitions can be found in my book "Sentiment Analysis and Opinion Mining". They are based on several of our papers in 2004 and 2005. The abstraction provides a model of online opinions, describes what should be extracted from opinion sources (e.g., reviews, forums, and blogs) and how the results may be organized and presented to the user. The main mining tasks are:

3. Sentiment Analysis of Comparative Opinions

A comparative sentence usually expresses an ordering relation between two sets of entities with respect to some shared features (or aspects). For example, the comparative sentence "Canon's optics are better than those of Sony and Nikon" expresses the comparative relation: (better, {optics}, {Canon}, {Sony, Nikon}). Comparative sentences use different language constructs from typical opinion sentences (e.g., "Cannon's optic is great").

Abstraction of the problem: "which is better than which on what". Again, the formal definitions can be found in my book "
Sentiment Analysis and Opinion Mining". The main mining tasks are:

Opinion Lexicon (or Sentiment Lexicon)

Data Sets


Publications - (sentiment analysis)                Publications - (opinion spam or fake review detection)

  1. Zhiyuan Chen, Arjun Mukherjee, Bing Liu, Meichun Hsu, Malu Castellanos, and Riddhiman Ghosh. Exploiting Domain Knowledge in Aspect Extraction. To appear in Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-2013), October 18-21, 2013, Seattle, USA.

  2. Arjun Mukherjee, Vivek Venkataraman, Bing Liu, and Sharon Meraz. Public Dialogue: Analysis of Tolerance in Online Discussions. To appear in Proceedings of The 51st Annual Meeting of the Association for Computational Linguistics (ACL-2013), August 4-9, 2013, Sofia, Bulgaria.

  3. Arjun Mukherjee, Bing Liu. Discovering User Interactions in Ideological Discussions. To appear in Proceedings of The 51st Annual Meeting of the Association for Computational Linguistics (ACL-2013), August 4-9, 2013, Sofia, Bulgaria.

  4. Jianfeng Si, Arjun Mukherjee, Bing Liu, Qing Li, Huayi Li, and Xiaotie Deng. Exploiting Topic based Twitter Sentiment for Stock Prediction. To appear in Proceedings of The 51st Annual Meeting of the Association for Computational Linguistics (ACL-2013, short paper), August 4-9, 2013, Sofia, Bulgaria.

  5. Zhiyuan Chen, Bing Liu, Meichun Hsu, Malu Castellanos, and Riddhiman Ghosh. Identifying Intention Posts in Discussion Forums. Proceedings of The 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT-2013), June 9-15, 2013, Atlanta, USA.

  6. Bing Liu. Sentiment Analysis and Opinion Mining. Morgan & Claypool Publishers, May 2012.

  7. Arjun Mukherjee and Bing Liu. Modeling Review Comments. Proceedings of 50th Annual Meeting of Association for Computational Linguistics (ACL-2012), July 8-14, 2012, Jeju, Republic of Korea.

  8. Arjun Mukherjee and Bing Liu. Aspect Extraction through Semi-Supervised Modeling. Proceedings of 50th Annual Meeting of Association for Computational Linguistics (ACL-2012), July 8-14, 2012, Jeju, Republic of Korea.

  9. Arjun Mukherjee and Bing Liu. Mining Contentions from Discussions and Debates. to appear in Proceedings of SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2012), Aug. 12-16, 2012, Beijing, China.

  10. Lei Zhang and Bing Liu. "Extracting Resource Terms for Sentiment Analysis," Proceedings of the 5th International Joint Conference on Natural Language Processing (IJCNLP-2011), November 8-13, 2011, Chiang Mai, Thailand.

  11. Zhongwu Zhai, Bing Liu, Lei Zhang, Hua Xu, Peifa Jia. Identifying Evaluative Opinions in Online Discussions. Proceedings of AAAI-2011, San Francisco, USA, August 7-11, 2011.

  12. Lei Zhang and Bing Liu. "Identifying Noun Product Features that Imply Opinions." ACL-2011 (short paper), Portland, Oregon, USA, June 19-24, 2011.

  13. Guang Qiu, Bing Liu, Jiajun Bu and Chun Chen. "Opinion Word Expansion and Target Extraction through Double Propagation." Computational Linguistics, March 2011, Vol. 37, No. 1: 9.27.

  14. Zhongwu Zhai, Bing Liu, Hua Xu, Peifa Jia. "Constrained LDA for Grouping Product Features in Opinion Mining." Proceedings of PAKDD-2011, Shenzhen, China, 2011. (Best Paper Award)

  15. Lei Zhang and Bing Liu. "Entity Set Expansion in Opinion Documents." Proceedings of the ACM Conference on Hypertext and Hypermedia (HT-2011), Eindhoven, Netherlands, June 6-9, 2011.

  16. Zhongwu Zhai, Bing Liu, Hua Xu and Peifa Jia. "Clustering Product Features for Opinion Mining." Proceedings of Fourth ACM International Conference on Web Search and Data Mining (WSDM-2011), Feb. 9-12, 2011, Hong Kong, China.

  17. Arjun Mukherjee and Bing Liu. "Improving Gender Classification of Blog Authors." Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-10). Oct. 9-11, 2010, MIT, Massachusetts, USA.

  18. Xiaowen Ding and Bing Liu. "Resolving Object and Attribute Coreference in Opinion Mining." Proceedings of the 23rd International Conference on Computational Linguistics (COLING-2010), August 23-27, Beijing, China.

  19. Zhongwu Zhai, Bing Liu, Hua Xu and Peifa Jia. "Grouping Product Features Using Semi-Supervised Learning with Soft-Constraints" Proceedings of the 23rd International Conference on Computational Linguistics (COLING-2010), August 23-27, Beijing, China.

  20. Lei Zhang and Bing Liu. "Extracting and Ranking Product Features in Opinion Documents." Proceedings of the 23rd International Conference on Computational Linguistics (COLING-2010), August 23-27, Beijing, China.

  21. Bing Liu. "Sentiment Analysis: A Multifaceted Problem." Invited paper, IEEE Intelligent Systems, 25(3), 2010, pp. 76-80.

  22. Bing Liu. "Sentiment Analysis and Subjectivity." Invited Chapter for the Handbook of Natural Language Processing, Second Edition. March, 2010.

  23. Ramanathan Narayanan, Bing Liu and Alok Choudhary. "Sentiment Analysis of Conditional Sentences." Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-09). August 6-7, 2009. Singapore.

  24. Guang Qiu, Bing Liu, Jiajun Bu and Chun Chen. "Expanding Domain Sentiment Lexicon through Double Propagation." Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI-09), Pasadena, California, USA, July 11-17, 2009.

  25. Xiaowen Ding, Bing Liu and Lei Zhang. "Entity Discovery and Assignment for Opinion Mining Applications," Proceedings of ACM SIGKDD Interntaional Conference on Knowledge Disocvery and Data Mining (KDD-09, industrial track), June 28-July 1, 2009, Paris.

  26. Bing Liu. "Opinion Mining." Invited contribution to Encyclopedia of Database Systems, 2008.

  27. Murthy Ganapathibhotla and Bing Liu. "Mining Opinions in Comparative Sentences." Proceedings of the 22nd International Conference on Computational Linguistics (Coling-2008), Manchester, 18-22 August, 2008.

  28. Xiaowen Ding, Bing Liu and Philip S. Yu. "A Holistic Lexicon-Based Appraoch to Opinion Mining." Proceedings of First ACM International Conference on Web Search and Data Mining (WSDM-2008), Feb 11-12, 2008, Stanford University, Stanford, California, USA.

  29. Xiaowen Ding and Bing Liu. "The Utility of Linguistic Rules in Opinion Mining." SIGIR-2007 (poster paper), 23-27 July 2007, Amsterdam.

  30. Nitin Jindal and Bing Liu. "Identifying Comparative Sentences in Text Documents" Proceedings of the 29th Annual International ACM SIGIR Conference on Research & Development on Information Retrieval (SIGIR-06), Seattle 2006.

  31. Nitin Jindal and Bing Liu. "Mining Comprative Sentences and Relations." Proceedings of 21st National Conference on Artificial Intellgience (AAAI-2006), July 16.20, 2006, Boston, Massachusetts, USA.

  32. Bing Liu, Minqing Hu and Junsheng Cheng. "Opinion Observer: Analyzing and Comparing Opinions on the Web"Proceedings of the 14th international World Wide Web conference (WWW-2005), May 10-14, 2005, in Chiba, Japan.

  33. Minqing Hu and Bing Liu. "Mining Opinion Features in Customer Reviews." Proceedings of Nineteeth National Conference on Artificial Intellgience (AAAI-2004), San Jose, USA, July 2004.

  34. Minqing Hu and Bing Liu. "Mining and summarizing customer reviews." Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-2004, full paper), Seattle, Washington, USA, Aug 22-25, 2004.

Publications - (Opinion spam or fake review detection)           (Check out my Opinion Spam Detection project homepage)

  1. Tieyun Qian, Bing Liu. Identifying Multiple Userids of the Same Author. To appear in Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-2013), October 18-21, 2013, Seattle, USA.

  2. Arjun Mukherjee, Abhinav Kumar, Bing Liu, Junhui Wang, Meichun Hsu, Malu Castellanos, and Riddhiman Ghosh. Spotting Opinion Spammers using Behavioral Footprints. To appear in Proceedings of SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2013), August 11-14 2013 in Chicago, USA.

  3. Geli Fei, Arjun Mukherjee, Bing Liu, Meichun Hsu, Malu Castellanos, and Riddhiman Ghosh. Exploiting Burstiness in Reviews for Review Spammer Detection. Proceedings of The International AAAI Conference on Weblogs and Social Media (ICWSM-2013), July 8-10, 2013, Boston, USA.

  4. Arjun Mukherjee, Vivek Venkataraman, Bing Liu, and Natalie Glance. What Yelp Fake Review Filter Might Be Doing. Proceedings of The International AAAI Conference on Weblogs and Social Media (ICWSM-2013), July 8-10, 2013, Boston, USA.

  5. Arjun Mukherjee, Bing Liu, and Natalie Glance. Spotting Fake Reviewer Groups in Consumer Reviews. International World Wide Web Conference (WWW-2012), Lyon, France, April 16-20, 2012.

  6. Guan Wang, Sihong Xie, Bing Liu, Philip S. Yu. Identify Online Store Review Spammers via Social Review Graph. ACM Transactions on Intelligent Systems and Technology, accepted for publication, 2011.

  7. Guan Wang, Sihong Xie, Bing Liu, Philip S. Yu. Review Graph based Online Store Review Spammer Detection. ICDM-2011, 2011.

  8. Arjun Mukherjee, Bing Liu, Junhui Wang, Natalie Glance, Nitin Jindal. Detecting Group Review Spam. WWW-2011 poster paper, 2011.

  9. Ee-Peng Lim, Viet-An Nguyen, Nitin Jindal, Bing Liu and Hady Lauw. "Detecting Product Review Spammers using Rating Behaviors." The 19th ACM International Conference on Information and Knowledge Management (CIKM-2010, full paper), Toronto, Canada, Oct 26 - 30, 2010.

  10. Nitin Jindal, Bing Liu and Ee-Peng Lim. "Finding Unusual Review Patterns Using Unexpected Rules." The 19th ACM International Conference on Information and Knowledge Management (CIKM-2010, short paper), Toronto, Canada, Oct 26 - 30, 2010.

  11. Nitin Jindal and Bing Liu. "Opinion Spam and Analysis." Proceedings of First ACM International Conference on Web Search and Data Mining (WSDM-2008), Feb 11-12, 2008, Stanford University, Stanford, California, USA.

  12. Nitin Jindal and Bing Liu. "Review Spam Detection." Proceedings of WWW-2007 (poster paper), May 8-12, Banff, Canada.

Created on May 15, 2004 by Bing Liu; and Minqing Hu.