The ISC is part of the Johns Hopkins Applied Physics Laboratory and will follow all current policies. Please visit the JHU/APL page for more information on the Lab's visitor guidance.

2022

Curvature-informed multi-task learning for graph networks


Abstract

Properties of interest for crystals and molecules, such as band gap, elasticity, and solubility, are generally related to each other: they are governed by the same underlying laws of physics. However, when state-of-the-art graph neural networks attempt to predict multiple properties simultaneously (the multi-task learning (MTL) setting), they frequently underperform a suite of single property predictors. This suggests graph networks may not be fully leveraging these underlying similarities.  Here we investigate a potential explanation for this phenomenon – the curvature of each property’s loss surface significantly varies, leading to inefficient learning. This difference incurvature can be assessed by looking at spectral properties of the Hessians of each property’s loss function, which is done in a matrix-free manner via randomized numerical linear algebra. We evaluate our hypothesis on two benchmark datasets (Materials Project (MP) and QM8) and consider how these findings can inform the training of novel multi-task learning models.


Citation

@inproceedings new2022curvatureinformed title=Curvature-informed multi-task learning for graph networks author=Alexander New and Michael J Pekala and Nam Q Le and Janna Domenico and Christine D. Piatko and Christopher D Stiles booktitle=ICML 2022 2nd AI for Science Workshop year=2022 url=https://openreview.net/forum?id=m5RYtApKFOg

Citation

@inproceedings new2022curvatureinformed title=Curvature-informed multi-task learning for graph networks author=Alexander New and Michael J Pekala and Nam Q Le and Janna Domenico and Christine D. Piatko and Christopher D Stiles booktitle=ICML 2022 2nd AI for Science Workshop year=2022 url=https://openreview.net/forum?id=m5RYtApKFOg