June 25, 2021 (Afternoon; Pacific Time)

12:00 - 12:10 Welcoming Remark and Workshop Overview (Michael Black)

Session I (Chair: Natalia Neverova)

12:10 - 12:40 Invited Talk I: Mackenzie Mathis (EPFL)

Title: Robust and Efficient Animal Pose Estimation

Abstract: Neural networks are highly effective tools for pose estimation. I will discuss our work on robustness and leveraging transfer learning for animal pose estimation. In particular, I will discuss open challenges for robustness to out-of-domain data, and ongoing work to leverage optimized datasets for building more robust and body plan-agnostic sharable tools and models for the larger community of users across life sciences and computer vision.

Bio: Mackenzie Mathis is the Bertarelli Foundation Chair of Integrative Neuroscience at EPFL, and a lead-developer of DeepLabCut, the software package for animal pose estimation. Her lab works on understanding adaptive mechanisms in intelligent systems. She received her PhD at Harvard in 2017.

12:40 - 12:45 Paper Presentation I: Spatio-Temporal Event Segmentation for Wildlife Extended Videos

12:45 - 12:55 Q&A

Session III (Chair: Shohei Nobuhara)

03:00 - 03:30 Invited Talk IV: Kristin Branson (Howard Hughes Medical Institute)

Title: Can Self-supervised Machine Learning Help Us Discover Principles of Animal Behavior?

Abstract: In recent years, supervised machine learning approaches have made huge strides in solving important computer vision problems related to animal behavior analysis, including video-based tracking and action recognition. These approaches involve training classifiers to replicate human labels, with the goal of automating, at high-throughput, tasks humans perform well at. A new frontier is to try to use self-supervised machine learning to predict relationships between different types of data better than humans. In our work, we have been training classifiers that can predict animals' future behaviors from their pasts. Our goal is to discover principles of animal behavior by interpreting and interrogating these classifiers. In this talk, I will discuss our current progress toward these goals, and the open questions we are facing.

Bio: Kristin Branson is a Senior Group Leader and the Head of Computation and Theory at the Howard Hughes Medical Institute's Janelia Research Campus. Her research focuses on developing cutting-edge machine vision and learning methods that can advance biological insight. She has developed influential software for video-based tracking and categorization of animal behavior that has been adopted by hundreds of labs. Her lab is using these tools to understand the structure, organization, and neural implementation of behavior. As the Head of Computation and Theory, she is responsible for leading all computational research at Janelia. Branson received her undergraduate degree from Harvard, her PhD from University of California San Diego with Serge Belongie and Sanjoy Dasgupta, and performed her postdoctoral studies at Caltech with Pietro Perona and Michael Dickinson. She joined Janelia as a Group Leader in 2010.

03:30 - 03:35 Paper Presentation III: hSMAL: Detailed Horse Shape and Pose Reconstruction for Motion Pattern Recognition

03:35 - 03:40 Paper Presentation IV: Analysis of Visual Attention of a Harris’ Hawk in Flight using a Synthetic Reconstruction of its Visual Field

03:40 - 03:50 Q&A

Session IV (Chair: Angjoo Kanazawa)

03:50 - 03:55 Paper Presentation V: Towards Self-Supervision for Video Identification of Individual Holstein-Friesian Cattle: The Cows2021 Dataset

03:55 - 04:25 Invited Talk V: Marc Badger (UPenn)

Title: Birds of a Feather Flock Together in 3D Shape Space

Abstract: Automated understanding of animal activity is poised to transform many fields in biology. Of particular interest to ecology, biomechanics, behavior, and neuroscience are studies of animals moving through naturalistic 3D environments and interacting in complex social groups. In both situations, we aim to capture movement dynamics and social cues encoded in pose trajectories and shape changes (such as puffing and hair raising), while gracefully handling frequent occlusions that occur when animals interact with each other and objects in the environment. Working with socially gregarious cowbirds as a model species, we developed a method for single-view pose and shape estimation using an articulated 3D mesh model, allowing accurate reconstruction of a flock of birds interacting in a large aviary using relatively few cameras. In our most recent work, we disentangle pose and shape to capture shape spaces of novel bird and dog species directly from image collections, bypassing the need for multi-view setups or 3D scans of many individuals. We then formed a multi-species shape space, which reflects the phylogenetic relationships among birds better than learned perceptual features. I will also discuss the challenges involved with using such parametric models to study animal morphometrics and the challenges related to multi-view multi-animal tracking in settings where only limited annotations are available.

Bio: Marc Badger is a postdoctoral researcher in Computer Science at the University of Pennsylvania advised by Marc Schmidt and Kostas Daniilidis. His research is at the intersection of machine perception and animal locomotion, focusing on animal pose and shape estimation, tracking, and action recognition. He loves creating high-throughput robotic systems to examine the relationships between biomechanics, learning, and animal behavior. Marc received his PhD in Integrative Biology at UC Berkeley, where he studied hummingbird flight biomechanics and was advised by Robert Dudley, and received his Bachelor’s in Physics from Harvey Mudd College. He has also spent time with Stacey Combes at UC Davis working on maneuvering flight and obstacle avoidance in bees. His work has been funded by a Graduate Research Fellowship from the National Science Foundation and a CiBER IGERT Traineeship, and has been featured in National Geographic Magazine.

04:25 - 04:35 Q&A

04:35 - 04:40 Closing Remark (Hyun Soo Park)

04:40 - Additional Poster Session and Mingling (Gatherly)