자료유형  

 학위논문

Control Number  

0016931021

International Standard Book Number  

9798380621113

Dewey Decimal Classification Number  

621.3

Main Entry-Personal Name  

Lakshminarayanan, Aravind Srinivas.

Publication, Distribution, etc. (Imprint  

[S.l.] : University of California, Berkeley., 2021

Publication, Distribution, etc. (Imprint  

Ann Arbor : ProQuest Dissertations & Theses, 2021

Physical Description  

1 online resource(113 p.)

General Note  

Source: Dissertations Abstracts International, Volume: 85-04, Section: B.

General Note  

Advisor: Abbeel, Pieter.

Dissertation Note  

Thesis (Ph.D.)--University of California, Berkeley, 2021.

Restrictions on Access Note  

This item must not be sold to any third party vendors.

Summary, Etc.  

요약The goal of extracting reusable and rich representations that capture what you care about for downstream tasks remains challenging even though the field of deep learning has made tremendous progress in this direction. This thesis presents a few promising contributions to further that goal. The two axes of contributions are: (1) self-supervised (or unsupervised) representation learning; (2) deep neural network architectures powered by self-attention. Progress in architectures and the ability to leverage massive amounts of unlabeled data have been responsible for major advances in NLP such as GPT-x and BERT. This thesis presents small steps towards realizing such progress for perceptual and reinforcement learning tasks. This is a thesis by articles containing four articles, two focused on computer vision benchmarks, with the other two focused on reinforcement learning.With respect to the first axis, the thesis presents three articles: (1) Data-Efficient Image Recognition using Contrastive Predictive Coding (CPCv2); (2) Contrastive Unsupervised Representations for Reinforcement Learning (CURL); (3) Reinforcement Learning with Augmented Data (RAD). The first two articles explore a form of unsupervised learning called contrastive learning, a technique better suited for raw inputs such as images compared to generative pre-training that is popular for language. The first article presents results for label-efficient image recognition. The second article presents the benefits of contrastive learning for sample-efficient reinforcement learning from pixels. Contrastive learning in practice is heavily dependent on data augmentations, and the third article presents a detailed investigation and discussion of its role.As for the second axis, the thesis presents a thorough empirical investigation of the benefits of self-attention and Transformer-like architectures for computer vision through the article: Bottleneck Transformers for Visual Recognition. Self-attention has revolutionized language processing but computer vision presents a challenge to vanilla Transformers through high resolution inputs that challenge the quadratic memory and computational complexity of the primitive. The article presents the empirical effectiveness of a straightforward hybrid composed of convolutions and self-attention and unifies the ResNet and Transformer based architecture design for computer vision.

Subject Added Entry-Topical Term  

Computer engineering.

Subject Added Entry-Topical Term  

Information technology.

Index Term-Uncontrolled  

Computer vision

Index Term-Uncontrolled  

Deep learning

Index Term-Uncontrolled  

Reinforcement learning

Index Term-Uncontrolled  

Representation learning

Index Term-Uncontrolled  

Self-supervised learning

Added Entry-Corporate Name  

University of California, Berkeley Computer Science

Host Item Entry  

Dissertations Abstracts International. 85-04B.

Host Item Entry  

Dissertation Abstract International

Electronic Location and Access  

로그인을 한후 보실 수 있는 자료입니다.

Control Number  

joongbu:640888