Lifelong Learning Machines

Training and evaluating a lifelong learning agent requires the ability to generate a diverse range of challenges, in a variety of environments, with rich systematic variation.

• MetaArcade is a highly parameterized environment for constructing custom 2D arcade games with controlled variation of perceptual features, actions, and rewards (code, paper, video).
• L2Explorer is a Unity-based first-person-view 3D exploration environment that can be configured on the fly to generate a range of tasks and task variants that can be structured into complex and evolving evaluation curricula (code, paper, video).

Related Publications

• Johnson, E. C., E. Q. Nguyen, B. Schreurs, C. S. Ewulum, C. Ashcraft, N. M. Fendley, M. M. Baker, A. New, G. K. Vallabha, “L2Explorer: A Lifelong Reinforcement Learning Assessment Environment,” AAAI Spring Symposium 2022 (Designing AI for Open Worlds) (2022).
• Staley, E. W., C. Ashcraft, B. Stoler, J. Markowitz, G. Vallabha, C. Ratto, K. D. Katyal, “Meta Arcade: A Configurable Environment Suite for Meta-Learning,” Deep RL Workshop NeurIPS 2021.

Evaluating a lifelong learning agent requires characterizing the tasks encountered by the agent over its mission, identifying the capabilities of the agent (how well it can learn multiple tasks and exploit task relationships), and exercising the agent across different mission scenarios in a consistent yet flexible way.

• RLBlocks is a framework of building blocks for lifelong learning agents (code).
• TELLA is a framework for Training and Evaluating Lifelong Learning Agents (code, paper).
• CLAMP is an algorithm to identify the capabilities of a learning agent using its performance data (code, paper).
• L2Metrics is a Python library for calculating lifelong learning metrics from performance data (code, paper).

Related Publications

• Baker, M., et al., “A domain-agnostic approach for characterization of lifelong learning systems,” Neural Networks, 160, pp. 274–296 (2023).
• Fendley, N., C. Costello, E. Nguyen, G. Perrotta, C. Lowman “Continual Reinforcement Learning with TELLA,” Conference on Lifelong Learning Agents (CoLLAs) 2022, arxiv:2208.04287 (2022).
• Rivera, C., C. Ashcraft, A. New, J. Schmidt, G. Vallabha “Estimating Latent Properties of Lifelong Learning Systems,” Conference on Lifelong Learning Agents (CoLLAs) 2022, arxiv: 2207.14378 (2022).

Artificial agents struggle when asked to identify objects under different visual conditions, act upon input observations that are superficially different (such as different lighting conditions in natural imagery), or adapt to significant changes to their given task.

Related Publications

• Emmanuel, P. R., C. R. Ratto, N. G. Drenkow, J. J. Markowitz, J. J., “Semi-supervised domain transfer for robust maritime satellite image classification,” Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications V, Vol. 12538, pp. 296–309), SPIE (June 2023).
• Markowitz, J., R. Sheffield, G. Mullins, “Maritime Platform Defense with Deep Reinforcement Learning.” SPIE AI/ML for Multi-Domain Operations IV (2022).