Lifelong Learning - learn as "humans do"

Accumulating knowledge learned in the past and
using it to learn more knowledge

New book "Lifelong Machine Learning." by Z. Chen and B. Liu, Morgan & Claypool Publishers, November 2016.

Lifelong Machine Learning Tutorial. (Title: lifelong machine learning and computer reading the Web), KDD-2016, August 13-17, 2016, San Francisco, USA.

Lifelong Machine Learning Tutorial, IJCAI-2015, July 25-31, 2015, Buenos Aires, Argentina

Statistical learning algorithms like deep NN, SVM, HMM, CRF, and topic modeling have been very successful in machine learning and data mining applications. Given a dataset, such an algorithm simply runs on the dataset to produce a model without considering any related information or past learning results. Although these algorithms can still be improved, such single task and isolated algorithmic approaches to machine learning have their limits, e.g., it needs a large number of training examples and is only suitable for well-defined and narrow tasks. Looking ahead, the question is how to deal with these limitations to improve machine learning further. I believe the answer is lifelong machine learning or simply lifelong learning, which tries to mimic "human learning" (I use quotes because we don't know how humans learn)). The key characteristic of "human learning" is that humans learn continuously - we retain the knowledge gained from the past learning and use the knowledge to help future learning and problem solving. Existing isolated machine learning algorithms are not capable of doing that. However, without the lifelong machine learning capability, AI systems and machine learning will probably never be truly intelligent. We believe that now is the right time to explore lifelong learning. Big data offers a golden opportunity for lifelong learning because its large volume and diversity (very important) give us abundant information for discovering rich and commonsense knowledge automatically, which can enable an intelligent learning agent to perform continuous learning, to accumulate knowledge learned previously, and to become more and more knowledgeable and better and better at learning.

In my life, I don't remember that anyone has ever given me 1000 positive and 1000 negative examples and asked me to build a classifier of any kind. Since I have accumulated so much knowledge in the past, I can learn mostly with little effort and few examples. If I don't have the accumulated knowledge, even if you give 2000 training examples, I will not be able to learn. For example, I don't understand Arabic. If you give me 2000 Arabic documents and ask me to build a classifier, I cannot do it (without using a dictionary).

Transfer learning, multitask learning, and lifelong learning

Our Work


Created on Sep 24, 2014 by Bing Liu.