자료유형  

 학위논문

Control Number  

0017164532

International Standard Book Number  

9798384045496

Dewey Decimal Classification Number  

004

Main Entry-Personal Name  

Castro, Santiago.

Publication, Distribution, etc. (Imprint  

[S.l.] : University of Michigan., 2024

Publication, Distribution, etc. (Imprint  

Ann Arbor : ProQuest Dissertations & Theses, 2024

Physical Description  

193 p.

General Note  

Source: Dissertations Abstracts International, Volume: 86-04, Section: A.

General Note  

Advisor: Mihalcea, Rada.

Dissertation Note  

Thesis (Ph.D.)--University of Michigan, 2024.

Summary, Etc.  

요약Videos have become an integral part of our daily lives, with a rapidly growing number on YouTube, Netflix, and TikTok serving as testimony to their widespread popularity. Behind the simplicity of their interfaces and user experiences, the systems that power these products employ numerous video-understanding techniques, even for straightforward use cases such as finding a video on how to cook salmon. Despite the significant progress achieved in this area, there remains a gap between lab-setting capabilities and reality, as multiple phenomena are not adequately designed for realistic settings, causing various issues such as domain mismatches and the diverse way people interact in videos (e.g., sarcastically). My work aims to bridge this gap by enabling the understanding of video content in realistic settings.The issues that make current video understanding research unsuitable for real life can be classified into data, methods, and evaluation. The data aspect is crucial since current research has predominantly overlooked real-life settings. I present new datasets and benchmarks for such domains: daily situations and in-the-wild scenarios. These benchmarks measure the effectiveness of new methods in these more realistic settings. Likewise, I introduce a novel framework that accounts for a typical yet understudied human behavior: sarcasm. Sarcasm is particularly suited to be studied in video since I show that leveraging what we see and hear (as people commonly do) allows one to understand it better. For the methods aspect, I consider a fundamental issue, which is the impracticality and lack of scalability of the traditional in-the-lab setting, tuning one model for each newly addressed task and domain. I propose a robust method that allows practitioners to employ a single model for novel tasks and domains with satisfactory performance. Additionally, I present a technique to improve the compositional generalization of existing models. Finally, I focus on current practices for evaluation and propose a framework better suited to realistic settings. Current benchmarks for short video understanding have drawbacks, such as employing easy-to-detect distractor answers, not accounting for diversity when depicting the same situation, and not considering realistic settings. I present a novel evaluation format that tackles all these issues and a benchmark that leverages it. The benchmark shows a gap between the performance of several methods and humans.

Subject Added Entry-Topical Term  

Computer science.

Subject Added Entry-Topical Term  

Computer engineering.

Subject Added Entry-Topical Term  

Web studies.

Subject Added Entry-Topical Term  

Information technology.

Index Term-Uncontrolled  

Video understanding

Index Term-Uncontrolled  

Natural Language Processing

Index Term-Uncontrolled  

Computer Vision

Index Term-Uncontrolled  

Compositional generalization

Index Term-Uncontrolled  

Sarcasm

Added Entry-Corporate Name  

University of Michigan Computer Science & Engineering

Host Item Entry  

Dissertations Abstracts International. 86-04A.

Electronic Location and Access  

로그인을 한후 보실 수 있는 자료입니다.

Control Number  

joongbu:653984