ECE 57000: Artificial Intelligence (Fall 2022)

Important Links

• Syllabus (pdf), tentative course schedule (pdf), and prerequisite quiz (ungraded, also on Gradescope)
• Brightspace (Grades) - For the first time logging into Piazza, Gradescope, or Circuit, please click the links in Brightspace under “Content” and then under the module “Piazza, Gradescope, and Circuit Links”. This will help link your Purdue account with these external learning tools. After the first time, you can just use the links below.
• Piazza (Announcements and discussion)
• Gradescope (Quizzes and assignment submission)
• Circuit (Project checkpoint submission and peer reviews)
• Google Colab (Free computing environment including GPUs)

Assignments / Project Checkpoints

• Course project instructions
  • Checkpoint 1 - Draft review of selected research papers (at least 3)
  • Checkpoint 2 - Preliminary implementation and results
  • Logistics for final project deliverables

Optional textbooks

The bracketed acronym is used for referencing these books.

1. [DL] Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville, 2016. http://www.deeplearningbook.org
2. [ML] Machine Learning: A Probabilistic Perspective by Kevin P. Murphy, 2012. https://ebookcentral.proquest.com/lib/purdue/detail.action?docID=3339490
3. [PY] Python Data Science Handbook by Jake VanderPlas, 2016. https://jakevdp.github.io/PythonDataScienceHandbook/

Lecture content by week

1. Week 1 (8/22/2022) - Introduction to artificial intelligence
   • Monday: Introduction to AI. See syllabus, course schedule, and course project links above.
   • Wednesday: Overview of AI
   • Friday: How to select research papers
2. Week 2 (8/29/2022) - PCA and linear algebra
   • Monday: Principal Components Analysis (PCA), Review of linear algebra (notebook, pdf); Related reading: DL, Ch.2
   • Wednesday: (continued)
   • Friday: See PCA notes; PCA demo (notebook, pdf )
3. Week 3 (9/5/2022) - Introduction to machine learning
   • Monday: Labor Day
   • Wednesday: Intro. to ML; PCA generalization demo (notebook, pdf); Optional related reading: DL, Ch. 5.1
   • Friday: K-nearest neighbors (KNN) and evaluating ML methods, (pdf); KNN Demo (notebook, pdf); Optional related reading: KNN Classifier Notes, DL, Ch. 5.2-5.3
4. Week 4 (9/12/2022) - Linear models and gradient descent
   • Monday: (KNN continued) Linear and Logistic Regression; Optional related reading: PY, Linear regression, DL, Ch. 5.1.4 (short), ML, Ch. 7 and Ch. 8 (in-depth)
   • Wednesday: Gradient Descent; Gradient descent demo (notebook, pdf); Optional related reading: ML, Ch. 8 section 8.3
   • Friday: Gradient descent continued; Loss functions and regularization
5. Week 5 (9/19/2022) - Basics of deep learning
   • Monday: Basics of deep learning
   • Wednesday: PyTorch and automatic differentiation (notebook, pdf)
   • Friday: Basics of convolutional neural networks (CNN); Convolutions demo (notebook, pdf); Optional related reading: DL, Ch. 9
6. Week 6 (9/26/2022) - CNNs and Clustering
   • Monday: Continue CNNs (see updated slides); CIFAR-10 demo with BatchNorm and residual networks (notebook, pdf)
   • Wednesday: Clustering; K-means clustering demo (notebook, pdf)
   • Friday: Spectral clustering (see clustering slides); Optional related reading: ML, Ch 25.4.3
7. Week 7 (10/3/2022) - Review of probability
   • Monday: Review of probability; Optional related reading: DL, Ch. 3, ML, Ch. 2
   • Wednesday: Review of probability (continued)
   • Friday: Review of probability (continued); Density estimation
8. Week 8 (10/10/2022) - Density estimation
   • Monday: October break
   • Wednesday: Density estimation (continued)
   • Friday: Density estimation (continued)
9. Week 9 (10/17/2022) - GMMs and Autoencoders
   • Monday: Gaussian Mixture Model and EM Algorithm; Optional related reading: ML, Ch. 11
   • Wednesday: Autoencoders and VAEs; Optional related reading: Recent introduction to VAEs by original authors (2019), Original VAE paper (2013), From Variational to Deterministic Autoencoders
   • Friday: Autoencoders and VAEs (continued)
10. Week 10 (10/24/2022) - Generative Adversarial Networks (GAN)
    • Monday: Generative Adversarial Networks (GAN); Optional related reading: Original GAN paper, Deep Convolutional GANs
    • Wednesday: GANs (continued)
    • Friday: Deep Convolutional GAN (DCGAN); DCGAN MNIST tutorial (notebook, pdf); Optional: Transposed convolution visualization (notebook, pdf
11. Week 11 (10/31/2022) - Generative Adversarial Networks (GAN)
    • Monday: DCGAN (continued); Wasserstein GAN; Optional related reading: Original Wasserstein GAN paper, Improved WGAN paper with gradient penalty, A Primer on Optimal Transport
    • Wednesday: Wasserstein GAN (continued)
    • Friday: Normalizing Flows; Change of variables demo (notebook, pdf)
12. Week 12 (11/7/2022) - Normalizing Flows
    • Monday: Normalizing Flows (continued)
    • Wednesday: Normalizing Flows (continued)
    • Friday: Iterative normalizing flows via destrutive learning
13. Week 13 (11/14/2022) - Special Topics
    • Monday: Distribution Alignment
    • Wednesday: Distribution Alignment (continued)
    • Friday: Diffusion Models
14. Week 14 (11/21/2022)
    • Monday: Diffusion Models (continued, see updated slides)
    • Wednesday: (Thanksgiving break)
    • Friday: (Thanksgiving break)
15. Week 15 (11/28/2022)
    • Mon/Wed/Fri: Project Presentations
16. Week 16 (12/5/2022)
    • Mon/Wed/Fri: Project Presentations