/Few-Shot Human Motion Prediction via Meta-learning

Few-Shot Human Motion Prediction via Meta-learning

Liang-Yan Gui, Yuxiong Wang, Deva Ramanan and Jose M. F. Moura
Conference Paper, European Conference on Computer Vision (ECCV), October, 2018

Download Publication (PDF)

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.


Human motion prediction, forecasting human motion in a few milliseconds conditioning on a historical 3D skeleton sequence, is a long-standing problem in computer vision and robotic vision. Existing forecasting algorithms rely on extensive annotated motion capture data and are brittle to novel actions. This paper addresses the problem of few-shot human motion prediction, in the spirit of the recent progress on few-shot learning and meta-learning. More precisely, our approach is based on the insight that having a good generalization from few examples relies on both a generic initial model and an effective strategy for adapting this model to novel tasks. To accomplish this, we propose proactive and adaptive meta-learning (PAML) that introduces a novel combination of model-agnostic meta-learning and model regression networks and unifies them into an integrated, end-to-end framework. By doing so, our meta-learner produces a generic initial model through aggregating contextual information from a variety of prediction tasks, while effectively adapting this model for use as a task-specific one by leveraging learning-to-learn knowledge about how to transform few-shot model parameters to many-shot model parameters. The resulting PAML predictor model significantly improves the prediction performance on the heavily benchmarked H3.6M dataset in the small-sample size regime.

BibTeX Reference
author = {Liang-Yan Gui and Yuxiong Wang and Deva Ramanan and Jose M. F. Moura},
title = {Few-Shot Human Motion Prediction via Meta-learning},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2018},
month = {October},
publisher = {Springer},
keywords = {human motion prediction, few-shot learning, meta-learning},