Fall 2024 GRASP Seminar: Bob Collins & Yanxi Liu, Penn State University, “From Vision to Dynamics”
October 17 @ 2:00 pm - 3:00 pm
*This seminar will be held in-person in Levine 307 as well as virtually via Zoom.
ABSTRACT
In the realms of health and sports, quantitative analysis of human movement provides guidance for individualized performance characterization, training, and health assessment. Data collection for biomechanical analysis of human motion is typically performed using expensive, specialized hardware that requires experiments to occur in a constrained lab setting. Computer vision algorithms using inexpensive, off-the-shelf video cameras for data collection would enable observation in more natural environments with minimal cost. Towards this end, this talk reports on:
• Our effort in collecting the first publicly available Taiji-MultiModal (PSU TMM100) dataset containing 100 sequences of simplified 24-form Taiji with synchronized mocap, video, and foot-insole pressure maps.
• An unsupervised learning method for training a light-weight encoder suitable for 3D body pose classification and sequence-to-sequence temporal alignment.
• The first deep learning baseline that demonstrates reliable and repeatable mapping from either a single frame or short sequence of human pose (kinematics) into predicted foot pressure map output (dynamics), leading to image-based stability monitoring in natural environments.
Presenter
Bob Collins & Yanxi Liu - Learn More
Dr. Collins co-directs the Laboratory for Perception, Action, and Cognition (LPAC) in the CSE department at Penn State University (PSU). His research area is computer vision, with an emphasis on video scene understanding, automated surveillance, human activity modeling, and real-time tracking. He has been PI/co-PI of numerous NSF grants on topics in persistent object tracking, detection and tracking of pedestrians, human pose estimation, analysis of social behavior in crowds, and recently, vision-based inference of human dynamics. He received his PhD degree in Computer Science from the University of Massachusetts at Amherst. Dr. Collins was an associate research professor at CMU RI from 1996 to 2005, working on projects ranging from the DARPA Video Surveillance and Monitoring (VSAM) project to the 30-camera CBS/CMU EyeVision system broadcast live during SuperBowl XXXV.
Dr. Liu is a professor of EECS at Penn State University (PSU), University Park, USA, trained in physics/EE, computer science and theoretical robotics/AI (B.S, China; Ph.D. USA; Postdoc, France). With an NSF (USA) research-education fellowship award, she spent one year at DIMACS (NSF center for DIscrete MAthematics and Theoretical Computer Science) before joining the faculty of the Robotics Institute, Carnegie Mellon University for ten years. Currently at PSU, she is the director of the Human Motion Capture Lab for Smart Health and co-directs the Lab for Perception, Action and Cognition (LPAC). Dr. Liu has been a visiting professor at Stanford University, ETH Zurich, Tsinghua University and Google/MSR/MSRA. She is on sabbatical 2023-2024 at CMU. A central theme of Dr. Liu’s research is on group theory-based “computational regularity” for multimodality data (funded continuously by US NSF, including a prestigious multidisciplinary INSPIRE grant) with diverse applications in robotics, human/machine perception, human activity in sports and in health. Dr. Liu chaired three international competitions at CVPR, ECCV, ICCV on Computer Vision algorithms for Detecting Symmetry in the Wild, is the lead author for the book on “Computational Symmetry in Computer Vision and Computer Graphics”, and the lead editor for the book on “Computer Vision for Biomedical Image Applications”, as well as two granted US patents with Google and Microsoft respectively. She has served as a program chair for Computer Vision and Pattern Recognition (CVPR) Conference 2017 and Winter Conference on Applications of Computer Vision (WACV) 2019, area chair for all major computer vision/graphics conferences (CVPR/ECCV/ICCV/MICCAI/ACM MM/SIGGRAPH), and as an associate editor for IEEE Transaction of Pattern Analysis and Machine Intelligence (PAMI).