---
title: Week 2
---

Oct 02
: **Lecture**{: .label .label-green }[Transformer Architecture](#)   
  - Sequence Modeling and Early Language Models
  - Recurrent Neural Networks
  - Transformers
  - Pretraining and Fine-Tuning


Oct 04
: **Lecture**{: .label .label-green }[Hardware Aware Algorithm Design](#)
  - Introduction to Processors, Parallelism, and the GPU Memory Hierarchy
  - Introduction to Arithmetic Intensity and Measures of Efficiency
  - Hardware Aware Algorithms for Matrix Multiplication
  - Understanding Attention Efficiency Bottlenecks 
  - Hardware Aware Algorithms for Attention