Lifelong and Continual Learning

An AI system without this capability is never intelligent

Second Edition: "Lifelong Machine Learning." by Z. Chen and B. Liu, Morgan & Claypool, August 2018 (1st edition, 2016).

Added three new chapters: (4) Continual Learning and Catastrophic Forgetting, (5) Open-world Learning, (8) Continuous Knowledge Learning in Chatbots

Introduced the concept of learning on the job or learning while working.

Updated and/or reorganized the other chapters.

Download the first edition, Lifelong Machine Learning, Nov 2016.

An Interview in Nature Outlook, July 20, 2022.

Continual or Lifelong Learning

The traditional machine learning paradigm operates in isolation, where a learning algorithm is applied to a dataset to generate a model without considering previously acquired knowledge. However, this paradigm lacks the ability to learn continuously or incrementally, accumulate knowledge over time, and use the previously learned knowledge to help current or learning. It is now common knowledge that continaul machine learning or simply continual learning (a.k.a lifelong learning) is necessary for Artificial General Intelligence (AGI). Two key challenges are:

Catastrophic Forgetting (CF): While previously a significant challenge, it is now nearly resolved with the help of large pre-trained models (see .
Knowledge Transfer (KT): Numerous existing techniques have demonstrated good knowledge transfer capabilities. However, we should note that knowledge transfer has limitless potential and can continually be improved.

NSF Project - Cruise - Cross-system Architecture Design for Automous Wireless Networks based on Lifelong Machine Learning.

Tutorials, Short Courses and Survey

New Survey: Continual Learning of Natural Language Processing Tasks: A Survey. arXiv:2211.12701, 11/23/2022.
Continual Learning Dialogue Systems - Learning during Conversation. Tutorial @ SIGIR-2022, Madrid | July 11-15, 2022. (Sahisnu Mazumder and Bing Liu)
Lifelong and Continual Learning. A Short PhD Course (8 hours), Aalborg University, June 14 and 16, 2022. (Bing Liu and Zixuan Ke)
Continual Learning Dialogue Systems - Learning on the Job after Model Deployment. Tutorial (Aug. 20) @ IJCAI-2021 August 21-26, 2021, Montreal, Canada. (Sahisnu Mazumder and Bing Liu)
Lifelong Machine Learning Tutorial. Title: lifelong machine learning and computer reading the Web, KDD-2016, August 13-17, 2016, San Francisco, USA.
Lifelong Machine Learning Tutorial, IJCAI-2015, July 25-31, 2015, Buenos Aires, Argentina.

Keynote and Invited Talks

Autonomous Learning Agents: A Theoretical Framework. (forthcoming) Keynote talk @ International Conference on Artificial Intelligence & Machine learning (AIM-2024), June 10-12, 2024, San Francisco, CA, USA. Two related papers: paper 1 and paper 2.
Continual Learning in the Open World. Keynote talk @ The first International Conference on Machine Learning Algorithms. Feb. 23 – 24, India.
Unifying Continual Learning, OOD Detection and Open World Learning. Keynote talk @ Second Conference on Lifelong Learning Agents (CoLLAs-2023), Montreal, August 22-25, 2023.
Continual Learning: Theory and Algorithms. Invited talk @ International Congress on Basic Sciences, Beijing, July 21, 2023.
Unifying Continual Learning, OOD Detection and Open World Learning. Invited talk @ Institute of Automation and Institute of Computer Technology, Beijing, July 19, 2023.
AI Autonomy: Open World Continual Learning. Invited Talk @ JSAI 2023 Special Session on “Recent Trends in Open-world Continual Learning” Japan, June 6, 2023.
Autonomous AI: Open World Learning and Adaptation. Invited talk @ Peking University, May 26, 2023.
Theory and Algorithms for Continual Learning Invited talk @ Peking University, May 24, 2023.
Unifying Continual Learning and OOD Detection. Invited talk @ Grab Technology Company, Singapore, Feb. 22, 2023.
Unifying Continual Learning and OOD Detection. Invited talk @ Agency for Science, Technology and Research (A*STAR), Singapore, Feb. 21, 2023.
Unifying Continual Learning and OOD Detection. Distinguished Speaker - invited talk @ Institute of Data Science, National University of Singapore, Feb. 16, 2023.
Theory and Algorithms for Open-World Continual Learning. Keynote talk @ ATAL Faculty Development Program on “Social Media and Social Network Data Mining (SMSNDM)”. India. Jan. 7, 2023.
Continual Learning: Theory and Algorithms. Invited talk @ Shenzhen University, Dec. 16, 2022.
Theory and Algorithms for Open World Continual Learning. Keynote talk @ IEEE Inter. Conf. on Cloud Computing and Intelligent Systems (CCIS-2022), Nov. 27, 2022.
Continual Learning: From Theory to Algorithms. Invited talk @ CCF BigBdata 2022, Nov. 18-20, 2022.
Continual Learning of Natural Language Processing Tasks. Invited talk @ CDSC-WEST-2022, Nov. 1, 2022.
Autonomous AI: Self-Initiated Continual Learning in the Open World. Invited talk @ CIIS Open-world Learning Forum, Sept. 18, 2022.
Autonomous Machine Learning: Continual Learning in the Open World. Invited talk @ Intel Labs, July 25, 2022.
AI Autonomy: Pre- and Post-deployment Continual Learning. Invited talk @ PyData Chicago, June 30, 2022.
Post-deployment Contiunal Learning. Invited talk @ CVPR workshop - CLVision: Workshop on Continual Learning in Computer Vision (3rd Edition), June 20, 2022.
Continual Learning in Pre- and Post-Deployment. Invited talk @ Megagon Labs, June 10, 2022.
Batch and Online Continual Learning and Beyond. Invited talk @ Zhenjiang Labs, May 26, 2022.
AI Autonomy: Continual Learning on the Job. Distinguished research talk @ Amazon Alexa, Mar. 4, 2022.
Self-Motivated and Self-Supervised Open-World Continual Learning. Invited talk @ Mind & Machine Intelligence Summit @ UCSB, Feb. 16-17, 2022.
Self-Initiated Continual Learning for Autonomous Agents. Keynote talk @ The 16th International Conference on Intelligent Systems and Knowledge Engineering (ISKE 2021), Nov. 27, 2021.
Self-Initiated Open World Learning for Autonomous Agents. Talk @ A DARPA Sail-On Program meeting. Oct 29, 2021.
Self-motivated Continual Learning for Knowledge Accumulation. Invited talk @ NeSy-2021 Continual Learning Session, Oct. 25, 2021.
Continual and On-the-Job Learning. Invited talk @ IJCAI-2021 Workshop on Continual Semi-supervised Learning, Aug.19-20, 2021.
Continual and Interactive Learning after Model Deployment. Invited talk @ Baidu Research, July 27, 2021.
Continual and Interactive Learning after Model Deployment. Keynote talk @ International Conference on Data Intelligence and Knowledge Services, July 10, 2021.
Continual and Interactive Learning after Model Deployment. Invited talk @ Allen Institute for Artificial Intelligence (AI2), June 18, 2021.
Continual Learning Dialogue Systems - Learning after Model Deployment. Invited talk @ ICLR-21 Workshop on Neural Conversational AI, May 7, 2021.
Learning on the Job in the Open World. Invited talk @ Information Sciences Institute, Univesity of Southern California, Sept.11, 2020.
Learning on the Job in the Open World. Invited talk @ ICML-2020 Workshop on Continual Learning, July 17, 2020.

A Podcast: "Machines that Learn Like Humans" by my former student Zhiyuan Chen and Francesco Gadaleta (host).

A Resource Site maintained by Eric Eaton's group
DARPA program: Lifelong Learning Machines (L2M), 3/16/2017

Related Learning Paradigms: Transfer learning, multitask learning, and lifelong learning

Characterisitcs of lifelong learning: (1) learning continuously (ideally in the open world), (2) accumulating the previously learned knowledge to become more and more knowledgeable, (3) using the knowledge to learn more knowledge and adapting it for problem solving, (4) discovering new problems/tasks to be learned and learning them incrementally, and (5) learning on the job or learning while working, improving model during testing or model applications.
Transfer learning vs. lifelong learning: Transfer learning uses the source domain labeled data to help target domain learning. Unlike lifelong learning, transfer learning is not continual and has no knowledge retention (as it uses source labeled data, not learned knowledge). The source must be similar to the target (which are normally selected by the user). It is also only one-directional: source helps target, but not the other way around because the target has no or little labeled data.
Multitask learning vs. lifelong learning: Multitask learning jointly optimizes learning of multiple tasks. Although it is possible to make it continual, multitask learning does not retain any explicit knowledge except data, and when the number of task is really large, it is hard to re-learn everything when faced with a new task.

Publications
Book: Sahisnu Mazumder and Bing Liu. Lifelong and Continual Learning Dialogue Systems. Springer, February 2024.
Book: Zhiyuan Chen and Bing Liu. Lifelong Machine Learning. Morgan & Claypool, 2018 (2nd edition), 2016 (1st edition).

Haowei Lin, Yijia Shao, Weinan Qian, Ningxin Pan, Yiduo Guo, and Bing Liu. Class Incremental Learning Via Likelihood Ratio Based Task Prediction. Proceedings of Internaional Conference on Learning Representations (ICLR-2024), Vienna, Austria, May 7 - 11, 2024.
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Bing Liu. Learnability and Algorithm for Continual Learning. Proceedings of Fortieth International Conference on Machine Learning (ICML-2023), Jul 23 - 29th, 2023, Hawaii, USA.
Tatsuya Konishi, Mori Kurokawa, Chihiro Ono, Zixuan Ke, Gyuhak Kim, Bing Liu. Parameter-Level Soft-Masking for Continual Learning. Proceedings of Fortieth International Conference on Machine Learning (ICML-2023), Jul 23 - 29th, 2023, Hawaii, USA.
Yijia Shao, Yiduo Guo, Dongyan Zhao and Bing Liu. Class-Incremental Learning based on Label Generation. Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL-2023, short paper), Toronto, Canada, July 9-14, 2023.
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke, Bing Liu. Open-World Continual Learning: Unifying Novelty Detection and Continual Learning. arXiv:2304.10038 [cs.LG].
Yidou Guo, Bing Liu and Dongyan Zhao. Dealing with Cross-Task Class Discrimination in Online Continual Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 (CVPR-2023). Jun 18th - 22nd 2023, Vancouver, Canada 2023.
Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, Bing Liu. Continual Pre-training of Language Models. Proceedings of The Eleventh International Conference on Learning Representations (ICLR-2023), Kigali Rwanda, Mon May 1 — Fri May 5 2023.
Bing Liu, Sahisnu Mazumder, Eric Robertson, and Scott Grigsby. AI Autonomy: Self-Initiated Open-World Continual Learning and Adaptation. AI Magazine, May 21, 2023.
Zixuan Ke and Bing Liu. Continual Learning of Natural Language Processing Tasks: A Survey. arXiv:2211.12701 [cs.CL], Nov. 23, 2022.
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke and Bing Liu. A Theoretical Study on Solving Continual Learning. Proceedings of Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS-2022), Nov. 28 - Dec. 9, 2022.
Zixuan Ke, Haowei Lin, Yijia Shao, Hu Xu, Lei Shu and Bing Liu. Continual Training of Language Models for Few-Shot Learning. Proceedings of The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP-2022), December 7–11, 2022.
Gyuhak Kim, Zixuan Ke, and Bing Liu. A Multi-Head Model for Continual Learning via Out-of-Distribution Replay. Proceedings of Conference on Lifelong Learning Agents (CoLLAs 2022), August 22-24, 2022.
Yiduo Guo, Bing Liu and Dongyan Zhao. Online Continual Learning through Mutual Information Maximization. Proceedings of The 39th International Conference on Machine Learning (ICML-2022), Baltimore, Maryland USA July 17-23, 2022.
Gyuhak Kim, Sepideh Esmaeilpour, Changnan Xiao, Bing Liu. Continual Learning Based on OOD Detection and Task Masking. Proceedings of the CVPR-2022 Workshop on Continual Learning in Computer Vision, 2022.
Bing Liu, Sahisnu Mazumder, Eric Robertson, and Scott Grigsby. AI Autonomy: Self-Initiation, Adaptation and Continual Learning. arXiv:2203.08994 [cs.AI], March 17, 2022.
Bing Liu, Eric Robertson, Scott Grigsby, and Sahisnu Mazumder. Self-Initiated Open World Learning for Autonomous AI Agents. Proceedings of AAAI Symposium on 'Designing Artificial Intelligence for Open Worlds,' March 21-23, 2022.
Tatsuya Konishi, Mori Kurokawa, Chihiro Ono, Zixuan Ke, Gyuhak Kim, Bing Liu. Partially Relaxed Masks for Knowledge Transfer without Forgetting in Continual\ Learning. Proceedings of 26th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD-22)., MAY 16-19, 2022, Chengdu, China.
Yiduo Guo, Wenpeng Hu, Dongyan Zhao, Bing Liu. Adaptive Orthogonal Projection for Batch and Online Continual Learning. Proceedings of AAAI-2022 (virtual), Feb 21 - 28, 2022.
Bing Liu, Eric Robertson, Scott Grigsby, and Sahisnu Mazumder. Self-Initiated Open World Learning for Autonomous AI Agents. arXiv:2110.11385 [cs.AI], 2021.
Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, Lei Shu. Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning. Proceedings of Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS-2021), Dec 6th - 14th, 2021.
Qi Qin, Wenpeng Hu, Han Peng, Dongyan Zhao, Bing Liu. BNS: Building Network Structures Dynamically for Continual Learning. Proceedings of Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS-2021), Dec 6th - 14th, 2021.
Zixuan Ke, Bing Liu, Hu Xu and Lei Shu. CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks. Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP-2021), 7 – 11, November 2021, Punta Cana, Dominican Republic.

Zixuan Ke, Hu Xu and Bing Liu. Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks. Proceedings of Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-2021). Jun 6 - 11, 2021, Mexico City, Mexico.
Wenpeng Hu, Qi Qin, Mengyu Wang, Jinwen Ma, and Bing Liu. Continual Learning by Using Information of Each Class Holistically. Proceedings of AAAI-2021. 2021.

Bing Liu and Sahisnu Mazumder. Lifelong and Continual Learning Dialogue Systems: Learning during Conversation. Proceedings of AAAI-2021. 2021.
Zixuan Ke, Bing Liu, and Xingchang Huang. Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks. Proceedings of 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Dec. 6-12, 2020, Vancouver, Canada.

Sahisnu Mazumder, Bing Liu, Shuai Wang, and Sepideh Esmaeilpour. An Application-Independent Approach to Building Task-Oriented Chatbots with Interactive Continual Learning. NeurIPS-2020 Workshop on Human in the Loop Dialogue Systems (HLDS-2020). 2020.

Sahisnu Mazumder, Bing Liu, Nianzu Ma, Shuai Wang. Continuous and Interactive Factual Knowledge Learning in Verification Dialogues. NeurIPS-2020 Workshop on Human And Machine in-the-Loop Evaluation and Learning Strategies (HAMLETS-2020). 2020.

Bing Liu and Sahisnu Mazumder. Lifelong Learning Dialogue Systems: Chatbots that Self-Learn On the Job. arXiv:2009.10750 [cs.CL], Sept. 22, 2020.

Zixuan Ke, Bing Liu, Hao Wang, and Lei Shu. Continual Learning with Knowledge Transfer for Sentiment Classification. Proceedings of European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD-2020), Ghent, Belgium, 14-18, September 2020.

Bing Liu. Learning on the Job: Online Lifelong and Continual Learning. Proceedings of 34th AAAI Conference on Artifical Intelligence (AAAI-2020), Feb 7-12, 2020, New York City. (This work was done while I was on leave in Peking University).

Sahisnu Mazumder, Bing Liu, Shuai Wang, Nianzu Ma. Lifelong and Interactive Learning of Factual Knowledge in Dialogues. Proceedings of Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL-2019), 11-13 September 2019, Stockholm, Sweden.
Hao Wang, Bing Liu, Shuai Wang, Nianzu Ma and Yan Yang. Forward and Backward Knowledge Transfer for Sentiment Classification. Proceedings of The Eleventh Asian Conference on Machine Learning (ACML-2019), PMLR 101:457-472, 2019.
Hu Xu, Bing Liu, Lei Shu and P. Yu. Open-world Learning and Application to Product Classification. Proceedings of the Web Conference (formerly known as the WWW conference), San Francisco, May 13-17, 2019.
Wenpeng Hu, Zhou Lin, Bing Liu, Chongyang Tao, Zhengwei Tao, Jinwen Ma, Dongyan Zhao, Rui Yan. Overcoming Catastrophic Forgetting for Continual Learning via Model Adaptation. Proceedings of the Seventh International Conference on Learning Representations (ICLR-2019), New Orleans, Louisiana, May 6 – 9, 2019.
Guangyi Lv, Shuai Wang, Bing Liu, Enhong Chen, and Kun Zhang. Sentiment Classification by Leveraging the Shared Knowledge from a Sequence of Domains. Proceedings of the 24th International Conference on Database Systems for Advanced Applications (DASFAA-2019), April 22-25, 2019.
Shuai Wang, Guangyi Lv, Sahisnu Mazumder, Geli Fei, and Bing Liu. Lifelong Learning Memory Networks for Aspect Sentiment Classification. Proceedings of 2018 IEEE International Conference on Big Data (IEEE BigData 2018), Seattle, December 10-13, 2018.

Lei Shu, Hu Xu, and Bing Liu. Unseen Class Discovery in Open-world Classification. arXiv:1801.05609 [cs.LG], 18 Jan. 2018.

Sahisnu Mazumder, Nianzu Ma, and Bing Liu. Towards a Continuous Knowledge Learning Engine for Chatbots. arXiv:1802.06024 [cs.CL], 16 Feb. 2018. Previous title: "Towards an Engine for Lifelong Interactive Knowledge Learning in Human-Machine Conversations".

Hu Xu, Bing Liu, Lei Shu and Philip S. Yu. Lifelong Domain Word Embedding via Meta-Learning. Proceedings of International Conference on Artificial Intelligence (IJCAI-ECAI-2018). July 13-19 2018, Stockholm, Sweden.

Bing Liu. Lifelong Machine Learning: a Paradigm for Continuous Learning. Frontier Computer Science, 2017, 11(3): 359–361.

Lei Shu, Hu Xu, Bing Liu. DOC: Deep Open Classification of Text Documents. Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP-2017, oral presentation short paper), September 7–11, 2017, Copenhagen, Denmark.

Lei Shu, Hu Xu, and Bing Liu. Lifelong Learning CRF for Supervised Aspect Extraction. Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL-2017, oral presentation short paper), July 30-August 4, 2017, Vancouver, Canada.

Lei Shu, Bing Liu, Hu Xu, and Annice Kim. Lifelong-RL: Lifelong Relaxation Labeling for Separating Entities and Aspects in Opinion Targets. Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP-2016), November 1–5, 2016, Austin, Texas, USA.

Geli Fei, Shuai Wang, and Bing Liu. 2016. Learning Cumulatively to Become More Knowledgeable. Proceedings of SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2016), August 13-17, San Francisco, USA.

Geli Fei, and Bing Liu. 2016. Breaking the Closed World Assumption in Text Classification. Proceedings of NAACL-HLT 2016 , June 12-17, San Diego, USA.

Shuai Wang, Zhiyuan Chen, and Bing Liu. Mining Aspect-Speciﬁc Opinion using a Holistic Lifelong Topic Model. Proceedings of the International World Wide Web Conference (WWW-2016), April 11-15, 2016, Montreal, Canada.

Qian Liu, Bing Liu, Yuanlin Zhang, Doo Soon Kim and Zhiqiang Gao. Improving Opinion Aspect Extraction using Semantic Similarity and Aspect Associations. Proceedings of Thirtieth AAAI Conference on Artificial Intelligence (AAAI-2016), February 12–17, 2016, Phoenix, Arizona, USA.

Zhiyuan Chen, Nianzu Ma and Bing Liu. Lifelong Learning for Sentiment Classification. Proceedings of the 53st Annual Meeting of the Association for Computational Linguistics (ACL-2015, short paper), 26-31, July 2015, Beijing, China.

Zhiyuan Chen and Bing Liu. Mining Topics in Documents: Standing on the Shoulders of Big Data.. Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2014), August 24-27, New York, USA. [Code] [Dataset]

Zhiyuan Chen and Bing Liu. Topic Modeling using Topics from Many Domains, Lifelong Learning and Big Data. Proceedings of the 31st International Conference on Machine Learning (ICML 2014), June 21-26, Beijing, China.

Zhiyuan Chen, Arjun Mukherjee, and Bing Liu. Aspect Extraction with Automated Prior Knowledge Learning. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL 2014), June 22-27, 2014, Baltimore, USA.

Created on Sep 24, 2014 by Bing Liu.