ViT Transformers and CLIP - Part 1

Course Introduction

The Robotic AI Agent
Development Environment Setup
Mathematical Prerequisites

Foundations: Statistical Learning Theory

The learning problem
Linear Regression
Gradient Descent
Entropy
Maximum Likelihood Estimation
Binary Classification

Neural Networks

Feature extraction
Multiclass Classifier
Backpropagation
Regularization

Convolutional Neural Networks

How we understand scenes ?
Convolution and Correlation
CNN Architectures
Image Classification
What CNNs Learn
ResNets

Object Detection

Introduction to Object Detection.
Computer vision datasets
Region-based Object Detectors

Recursive State Estimation

Introduction to State Estimation with HMM.
The Bayes Filter
An example of Discrete Bayes Filter
Continuous state space and the Kalman filter
A Kalman filter example

Global Planning

Introduction to Planning.
Planning Domain Definition Language
Forward Search Algorithms
The A* Algorithm

Multimodal Reasoning and Transformers

Introduction to Transformers for language modeling.
The Learnable Attention Mechanism.
Multi-Head Self Attention
ViT Transformers and CLIP - Part 1

Markov Decision Processes

Introduction to MDPs - Part 1
Introduction to MDPs - Part 2
The Bellman Expectation Equations - Part 1
The Bellman Expectation Equations - Part 2
Policy Evaluation - Part 1
Policy Evaluation - Part 2
Bellman Optimal Value Functions
Policy Iteration and Value Iteration

AI for Robotics/Multimodal Reasoning and Transformers/ViT Transformers and CLIP - Part 1

ViT Transformers and CLIP Part 1

The best reference for studying ViT Transformers and CLIP is the free textbook: Foundations of Computer Vision - Chapter 26.

Please note that there is an overlap between this video and the previous youtube videos on Transformer attention mechanisms. Having said that this video introduces some more details.