Lifelong Learning - learn as "humans do"

Accumulate knowledge learned in the past and
use it to learn more knowledge to build a lifelong learning machine
(similar to Continuous Learning or Continual Learning)

New book "Lifelong Machine Learning." by Z. Chen and B. Liu, Morgan & Claypool Publishers, November 2016.

Lifelong Machine Learning Tutorial. Title: lifelong machine learning and computer reading the Web, KDD-2016, August 13-17, 2016, San Francisco, USA.

Lifelong Machine Learning Tutorial, IJCAI-2015, July 25-31, 2015, Buenos Aires, Argentina.

A Podcast: "Machines that Learn Like Humans" by my former student Zhiyuan Chen and Francesco Gadaleta (host).

A Resource Site maintained by Eric Eaton's group
DARPA new program Lifelong Learning Machines (L2M), 3/16/2017

Statistical learning algorithms like deep NN, SVM, HMM, CRF, and topic modeling have been very successful in machine learning and data mining applications. Given a dataset, such an algorithm simply runs on the dataset to produce a model without considering any related information or past learning results. Although these algorithms can still be improved, such single task and isolated algorithmic approaches to machine learning have their limits, e.g., it needs a large number of training examples and is only suitable for well-defined and narrow tasks. Looking ahead, the question is how to deal with these limitations to move machine learning to the next phase. I believe the answer is lifelong machine learning or simply lifelong learning (similar to continuous learning or even continual learning), which tries to mimic "human learning" (we don't know how humans learn) to build a lifelong learning machine. The key characteristic of "human learning" is the continuous learning and adaptation to new environments - we retain or accumulate the knowledge gained from past learning and use the knowledge to help future learning and problem solving with possible adaptations. Existing isolated machine learning algorithms are not capable of doing that. However, without the lifelong learning capability, AI systems will probably never be truly intelligent. We believe that now is the right time to explore lifelong learning. Big data offers a golden opportunity because its large volume and diversity give us abundant information for discovering rich and commonsense knowledge automatically, which can enable an lifelong learning machine or agent to continuously learn and accumulate knowledge, and to become more and more knowledgeable and better and better at learning.

Human learning is very different: I believe that no human being has ever been given 1000 positive and 1000 negative documents (or images) and asked to learn a text classifier. As we have accumulated so much knowledge in the past and understand it, we can usually learn with little effort and few examples. If we don't have the accumulated knowledge, even if we are given 2000 training examples, it is very hard to learn manually. For example, I don't understand Arabic. If you give me 2000 Arabic documents and ask me to build a classifier, I cannot do it. But that is exactly what the current machine learning algorithms do. That is not how humans learn.

Related Learning Paradigms: Transfer learning, multitask learning, and lifelong learning

Our Work


         TextBook: Zhiyuan Chen and Bing Liu. Lifelong Machine Learning. Morgan & Claypool, 2016.

Created on Sep 24, 2014 by Bing Liu.