Week 1 Jan 23
LectureIntroduction

TutorialHPC tutorial

  • History of self-driving cars
  • Introduction to embodied learning
Week 2 Jan 30
LectureDeep Learning for Structured Outputs

TutorialSimulator Tutorial

  • Object detection and segmentation
  • Graphical models
  • Energy-based models
  • Autoregressive models
Week 3 Feb 6
Lecture3D Vision, Mapping

TutorialVideo Learning Tutorial

  • Diffusion models
    • Probabilistic foundation
    • Applications in embodied learning
  • 3D network designs
    • Bird’s eye view networks
    • Point cloud networks
Week 4 Feb 13
LectureSelf-Supervised Representation Learning and Object Discovery

TutorialEgocentric Video Tutorial

  • 3D vision
    • Sensor fusion
    • Multi-task architecture
  • Physical grounding
    • Stereo, self-supervised depth
    • Optical flow
    • Unsupervised flow, depth and pose
  • Mapping
    • Soft mapping
    • Registration
  • Representation learning
    • DAE, MAE
    • Energy-based models
Week 5 Feb 20
LectureWorld Models and Forecasting

TutorialMotion Learning Tutorial

  • Representation learning
    • Energy-based models
    • Joint embedding models
  • Object discovery
    • Pseudo-labels
    • Slot-based models
    • Complex-valued autoencoders
  • World models
    • Trajectory prediction
    • Latent sequence models
    • Occupancy volume prediction
Week 6 Feb 27
LectureEnd-to-End Planning

TutorialLLM Agent Tutorial

  • World models
    • Latent prediction
    • Video prediction
    • 3D volume prediction
  • End-to-end planning
    • Imitation learning
    • Energy-based planning
    • Differentiable cost volume
    • Value-iteration networks
    • Backprop through planning
  • Continual learning
    • Parameter regularization
Week 7 Mar 6
LectureContinual Learning, Few-Shot Learning and Meta-Learning

SeminarDeep Learning for Structure Prediction

  • Continual learning
    • Variational continual learning
    • Knowledge distillation
    • Memory replay
    • Architectural expansion
    • Associative memory
    • Prompt learning
    • Continual self-supervised learning
  • Few-shot meta-learning
    • Learning to learn
    • Meta-optimization
    • MAML
    • Hypernetworks
    • Representation and memory
    • Continual few-shot learning
    • Few-shot skill learning
  • Seminar:
    • Tanishq Sardana: Segment Anything
    • Qing Mu: DETR: End-to-End Object Detection
    • Owais Saad Shuja: Latent Diffusion Models
Week 8 Mar 13
LectureGuest Lecture (Prof. Wei-Chiu Ma)

Seminar3D Vision

  • Seminar:
    • Sihang Li: Scene Coordinate Reconstruction
    • Kanishkha Jaisankar: NeRF
    • Denis Mbey Akola: DUSt3R
    • Zijin Hu: Zero-1-to-3
Week 9 Mar 20
SeminarSelf-Supervised Learning and World Models
  • Seminar:
    • Dahye Kim: DINOv2
    • Surbhi: IJEPA
    • Sal Yeung: Predictable and Robust Neural Representations by Straightening
    • Anurup Naskar: Moving Off-the-Grid
    • Andrew Deur: DayDreamer
    • Sidhartha Reddy Potu : UniSim
Week 11 Apr 3
SeminarWorld Models and End-to-End Planning
  • Seminar:
    • Sergey Sedov: DreamerV2 and Backpropagation-based Policy Gradients
    • Pratyaksh Prabhav Rao: DINO-WM
    • Rooholla Khorrambakht: Diffusion for World Modeling
    • Mrunal Sarvaiya: Differential MPC
    • Sushma Mareddy: MP3
    • Raman Kumar Jha: UniAD
    • Jovita Gandhi: Embodied GPT
Week 12 Apr 10
LectureGuest Lecture (Dr. Andrei Bârsan)
SeminarContinual Learning
  • Seminar:
    • Akshay Raman: Thinking Fast and Slow for Continual Learning
    • Amey Joshi: Continual Learning for Robotic Systems
    • Zifan Zhao: Loss of Plasticity in Deep Continual Learning
Week 13 Apr 17

SeminarFew-Shot Learning and LLM Agents

  • Seminar:
    • Ellen Su: Seeing the Un-Scene
    • Xu Zhang: FSL + Diffusion
    • Ravan Buddha: Gemini Robotics
    • Dan Zhao: Magma
    • Sunidhi Tandel: LEO
    • Solim LeGris: CoALA