Material Type  

 학위논문

 

0017162762

Date and Time of Latest Transaction  

20250211152051

ISBN  

9798382738338

DDC  

004

Author  

Louis, Nathan.

Title/Author  

Improving Articulated Pose Tracking and Contact Force Estimation for Qualitative Assessment of Human Actions.

Publish Info  

[S.l.] : University of Michigan., 2024

Publish Info  

Ann Arbor : ProQuest Dissertations & Theses, 2024

Material Info  

121 p.

General Note  

Source: Dissertations Abstracts International, Volume: 85-12, Section: B.

General Note  

Advisor: Corso, Jason J.;Owens, Andrew.

학위논문주기  

Thesis (Ph.D.)--University of Michigan, 2024.

Abstracts/Etc  

요약Using video to automate human performance metrics or skill analysis is an important but under-explored task. Currently, measuring the quality of an action can be highly subjective, where even assessments from experts are affected by bias and inter-rater reliability. In contrast, Computer vision and AI have the potential to provide real-time non-intrusive solutions with increased objectivity, scalability, and repeatability across various domains. From video alone, we can automatically provide supplemental objective scoring of Olympic sports, evaluate the technical skill of surgeons for training purposes, or monitor the physical rehabilitation progress of a patient. Today we solve these problems with supervised learning, obtaining features that represent high correlation with our desired point of analysis. Supervised learning is powerful, data-driven, and sometimes the best available option. However alone, it may be sub-optimal in the presence of scarce data and insufficient when needed to generalize to varying conditions or to truly understand the target task.In this dissertation, the bases of our human analysis understanding are skeletal poses, namely hand poses and full body poses. For articulated hand poses, we improve tracking using our CondPose network to integrate prior detection confidences and encourage tracking consistency. While for human poses, we propose two physical simulation-based metrics for evaluating physical plausibility and perform external force estimation through predicted ground reaction forces (GRFs). However, in the human analysis domain, collecting and annotating data at the scale of other deep learning tasks is a recurring challenge. This limits our generalizability to different environments, procedures, and motions. We address this by exploring semi-supervised learning methods, such as contrastive pre-training and multi-task learning.We apply articulated hand pose tracking in the surgical environment for assessing surgical skill. By applying a time-shifted sampling augmentation, we introduce clip-contrastive pre-training on embedded hand features as an unsupervised learning step. We show that this contrastive pre-training improves performance when fine-tuned on surgical skill classification and assessment task. Unlike most prior work, we evaluate on open surgery videos rather than solely simulated environments. Specifically, we use videos of non-laparoscopic, collected through collaboration with the Cardiac Surgery department at Michigan Medicine. We use full body poses and contact force estimation to bridge the gap between visual observations and the physical world. This physically-grounded component is vital for understanding actions involving sports or physical rehab where humans interact with their environment. We leverage multi-task learning to perform 2D-to-3D human pose estimation and integrate other abundant sources of motion capture data, without requiring additional force plate supervision. Our experiments shows that this improves GRF estimation on unseen motions. To address data limitations, we also collect two novel datasets SurgicalHands and ForcePose. We use SurgicalHands in the surgical domain as a multi-instance articulated hand pose tracking dataset. It encompasses a high degree of complexity in appearance and movement, not present in prior datasets. ForcePose is a multi-view GRF dataset of tracked human poses and time-synchronized force plates, to our knowledge the largest and most varied of its kind. This dataset serves as a benchmark for mapping human body motion and physical forces, enabling physical grounding of specific actions.

Subject Added Entry-Topical Term  

Computer science.

Subject Added Entry-Topical Term  

Electrical engineering.

Subject Added Entry-Topical Term  

Surgery.

Subject Added Entry-Topical Term  

Bioinformatics.

Index Term-Uncontrolled  

CondPose network

Index Term-Uncontrolled  

Physical rehabilitation

Index Term-Uncontrolled  

Multi-task learning

Index Term-Uncontrolled  

SurgicalHands

Index Term-Uncontrolled  

ForcePose

Added Entry-Corporate Name  

University of Michigan Electrical and Computer Engineering

Host Item Entry  

Dissertations Abstracts International. 85-12B.

Electronic Location and Access  

로그인을 한후 보실 수 있는 자료입니다.

Control Number  

joongbu:658647