Week 1 Jan 23
LectureIntroduction

TutorialHPC tutorial

  • History of self-driving cars
  • Introduction to embodied learning
Week 2 Jan 30
LectureDeep Learning for Structured Outputs

TutorialSimulator Tutorial

  • Object detection and segmentation
  • Graphical models
  • Energy-based models
  • Autoregressive models
Week 3 Feb 6
Lecture3D Vision, Mapping

TutorialVideo Learning Tutorial

  • Diffusion models
    • Probabilistic foundation
    • Applications in embodied learning
  • 3D network designs
    • Bird’s eye view networks
    • Point cloud networks
Week 4 Feb 13
LectureSelf-Supervised Representation Learning and Object Discovery

TutorialEgocentric Video Tutorial

  • 3D vision
    • Sensor fusion
    • Multi-task architecture
  • Physical grounding
    • Stereo, self-supervised depth
    • Optical flow
    • Unsupervised flow, depth and pose
  • Mapping
    • Soft mapping
    • Registration
  • Representation learning
    • DAE, MAE
    • Energy-based models
Week 5 Feb 20
LectureWorld Models and Forecasting

TutorialMotion Learning Tutorial

  • Representation learning
    • Energy-based models
    • Joint embedding models
  • Object discovery
    • Pseudo-labels
    • Slot-based models
    • Complex-valued autoencoders
  • World models
    • Trajectory prediction
    • Latent sequence models
    • Occupancy volume prediction
Week 6 Feb 27
LectureEnd-to-End Planning

TutorialLLM Agent Tutorial

  • World models
    • Latent prediction
    • Video prediction
    • 3D volume prediction
  • End-to-end planning
    • Imitation learning
    • Energy-based planning
    • Differentiable cost volume
    • Value-iteration networks
    • Backprop through planning
  • Continual learning
    • Parameter regularization
Week 7 Mar 6
LectureContinual Learning, Few-Shot Learning and Meta-Learning

SeminarDeep Learning for Structure Prediction

  • Continual learning
    • Variational continual learning
    • Knowledge distillation
    • Memory replay
    • Architectural expansion
    • Associative memory
    • Prompt learning
    • Continual self-supervised learning
  • Few-shot meta-learning
    • Learning to learn
    • Meta-optimization
    • MAML
    • Hypernetworks
    • Representation and memory
    • Continual few-shot learning
    • Few-shot skill learning
  • Seminar:
    • Segment Anything
    • DETR: End-to-End Object Detection
    • Latent Diffusion Models
Week 8 Mar 13
LectureGuest Lecture (Prof. Wei-Chiu Ma)

Seminar3D Vision

  • Seminar:
    • Scene Coordinate Reconstruction
    • NeRF
    • DUSt3R
    • Zero-1-to-3
Week 9 Mar 20
SeminarSelf-Supervised Learning and World Models
  • Seminar:
    • DINOv2
    • IJEPA
    • Predictable and Robust Neural Representations by Straightening
    • Moving Off-the-Grid
    • DayDreamer
    • UniSim
Week 11 Apr 3
SeminarWorld Models and End-to-End Planning
  • Seminar:
    • DreamerV2 and Backpropagation-based Policy Gradients
    • DINO-WM
    • Diffusion for World Modeling
    • Differential MPC
    • MP3
    • UniAD
    • Embodied GPT
Week 12 Apr 10
LectureGuest Lecture (Dr. Andrei Bârsan)
SeminarContinual Learning
  • Seminar:
    • Thinking Fast and Slow for Continual Learning
    • Continual Learning for Robotic Systems
    • Loss of Plasticity in Deep Continual Learning
Week 13 Apr 17

SeminarFew-Shot Learning and LLM Agents

  • Seminar:
    • Seeing the Un-Scene
    • FSL + Diffusion
    • Gemini Robotics
    • Magma
    • LEO
    • CoALA