Autonomous Learning: Learning on the Job
- Continual Learning after Deployment

Interactive AI and Lifelong Learning
Self-Initiated Learning through Interaction with Humans and the Environment


Second Edition: "Lifelong Machine Learning." by Z. Chen and B. Liu, Morgan & Claypool, August 2018 (1st edition, 2016)
Continual Learning Dialogue Systems - Learning on the Job after Model Deployment. Tutorial @ IJCAI-2021, August 21-26, 2021, Montreal, Canada.
Continual and Interactive Learning after Model Deployment. Invited talk @ Allen Institute for Artificial Intelligence (AI2), June 18, 2021.
Continual Learning Dialogue Systems - Learning after Model Deployment. Invited talk @ ICLR-21 Workshop on Neural Conversational AI, May 7, 2021.
Learning on the Job in the Open World. Invited talk @ Information Sciences Institute, Univesity of Southern California, Sept.11, 2020.
Learning on the Job in the Open World. Invited talk @ ICML-2020 Workshop on Continual Learning, July 17, 2020.
Lifelong Machine Learning Tutorial. Title: lifelong machine learning and computer reading the Web, KDD-2016, August 13-17, 2016, San Francisco, USA.
Lifelong Machine Learning Tutorial, IJCAI-2015, July 25-31, 2015, Buenos Aires, Argentina.

Motivation for Autonomous Learning: It is known that about 70% of our human knowledge comes from “on-the-job” learning. Only about 10% is learned through formal education and the rest 20% is learned through observation of others (imitation). An autonomous AI system must have this capability - its machine learning algorithm must be able to learn autonomously on the job or while working after model deployment. As the real world is too complex and constantly changing, it is impossible to learn everything through offline training using manually labeled data. An autonomous learning agent has to explore and learn by itself in the real world, which is open and constantly change - full of unknowns. AI agents must be able to detect the unknowns and learn them in a self-supervised manner through its interaction with humans, other agents, and the real-world environment. It should not make the closed-world assumption any more. Here are two motiviating examples:

Autonomous Learning: Like human on-the-job learning, it studies learning after model deployment or during model application (or testing) - after a good model has been built and deployed in an application. In classic machine learning, once a model is built, it is deployed in an application. During application, the model remains fixed or unchanged. Autonomous learning (or on-the-job learning) investigates continuous learning after model deployment, which involves the following steps

  1. continuously discover new tasks to learn by the agent itself. This is called Open World Learning or Out-of-Distribtuion Detection.
  2. gather "free" training data through the agent’s own active effort via interaction with humans, other agents and the environment.
  3. incrementally learn the new tasks without interrupting the application to become more and more knowledgeable. This is continual learning.
Here we emphesize interaction with humans and the environment to find new tasks and to naturally label training data (see the two motivating examples above).

Interactive AI: This is also called Interactive Self-Supervision. Step 2 is the key for on-the-job learning, i.e., how to find the hidden classes and obtain labeled training data. This must be done through actions initiated by the system itself without interrupting the application. That is, it must learn actively on its own based on its prior knowledge by observing and interaction with the environment and humans to obtain explicit or implicit feedback to serve as supervision. The interaction with humans should be through natural language dialogues. It is not possible for an autonomous intelligent agent to rely solely on massive manual labeled training data to learn passively offline forever.

Publications

         TextBook: Zhiyuan Chen and Bing Liu. Lifelong Machine Learning. Morgan & Claypool, 2018 (2nd edition), 2016 (1st edition).

Created on July 15, 2020 by Bing Liu.