ECE 57000: Artificial Intelligence (Fall 2021)

Important Links

• Syllabus (pdf), tentative course schedule (pdf), and prerequisite quiz (ungraded, also on Gradescope)
• Brightspace (Grades) - For the first time logging into Piazza, Gradescope, or Circuit, please click the links in Brightspace under “Content” and then under the module “Piazza, Gradescope, and Circuit Links”. This will help link your Purdue account with these external learning tools. After the first time, you can just use the links below.
• Piazza (Announcements and discussion)
• Gradescope (Quizzes and assignment submission)
• Circuit (Project checkpoint submission and peer reviews)
• Google Colab (Free computing environment including GPUs)

Assignments / Project Checkpoints

• Course project instructions
  • Checkpoint 1 - Draft review of selected research papers (at least 3)
  • Checkpoint 2 - Preliminary implementation and results
  • Logistics for final project deliverables
• Assignments - See Piazza for instructions and due dates

Optional textbooks

The bracketed acronym is used for referencing these books.

1. [DL] Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville, 2016. http://www.deeplearningbook.org
2. [ML] Machine Learning: A Probabilistic Perspective by Kevin P. Murphy, 2012. https://ebookcentral.proquest.com/lib/purdue/detail.action?docID=3339490
3. [PY] Python Data Science Handbook by Jake VanderPlas, 2016. https://jakevdp.github.io/PythonDataScienceHandbook/

Lecture content by week

1. Week 1 (8/23/2021) - Introduction to artificial intelligence
   • Monday: Introduction to AI. See syllabus, course schedule, and course project links above.
   • Wednesday: Overview of AI
   • Friday: How to select research papers
2. Week 2 (8/30/2021) - PCA and linear algebra
   • Monday: Principal Components Analysis (PCA), Review of linear algebra (notebook, pdf); Related reading: DL, Ch.2
   • Wednesday: (continued)
   • Friday: See updated PCA notes above; PCA demo (notebook, pdf )
3. Week 3 (9/6/2021) - Introduction to machine learning
   • Monday: Labor Day
   • Wednesday: Intro. to ML; PCA generalization demo (notebook, pdf); Optional related reading: DL, Ch. 5.1
   • Friday: K-nearest neighbors (KNN) and evaluating ML methods, (pdf); KNN Demo (notebook, pdf); Optional related reading: KNN Classifier Notes, DL, Ch. 5.2-5.3
4. Week 4 (9/13/2021) - Linear models and gradient descent
   • Monday: (KNN continued) Linear and Logistic Regression; Optional related reading: PY, Linear regression, DL, Ch. 5.1.4 (short), ML, Ch. 7 and Ch. 8 (in-depth)
   • Wednesday: Gradient Descent; Gradient descent demo (notebook, pdf); Optional related reading: ML, Ch. 8 section 8.3
   • Friday: Gradient descent continued; Loss functions and regularization
5. Week 5 (9/20/2021) - Basics of deep learning
   • Monday: Regularization continued; Basics of deep learning
   • Wednesday: PyTorch and automatic differentiation (notebook, pdf)
   • Friday: Basics of convolutional neural networks (CNN); Convolutions demo (notebook, pdf); Optional related reading: DL, Ch. 9
6. Week 6 (9/27/2021) - CNNs and Clustering
   • Monday: Continue CNNs (see updated slides); CIFAR-10 demo with BatchNorm and residual networks (notebook, pdf)
   • Wednesday: Finish CNNs, BatchNorm and residual networks; Clustering;
   • Friday: K-means clustering demo (notebook, pdf)
7. Week 7 (10/4/2021) - Review of probability
   • Monday: Spectral clustering (see clustering slides); Optional related reading: ML, Ch 25.4.3
   • Wednesday: Review of probability; Optional related reading: DL, Ch. 3, ML, Ch. 2
   • Friday: Review of probability (continued)
8. Week 8 (10/11/2021) - Density estimation
   • Monday: October break
   • Wednesday: Density estimation
   • Friday: Density estimation (continued)
9. Week 9 (10/18/2021) - GMMs and Autoencoders
   • Monday: Gaussian Mixture Model and EM Algorithm; Optional related reading: ML, Ch. 11
   • Wednesday: Autoencoders and VAEs; Optional related reading: Recent introduction to VAEs by original authors (2019), Original VAE paper (2013), From Variational to Deterministic Autoencoders
   • Friday: Autoencoders and VAEs (continued)
10. Week 10 (10/25/2021) - Generative Adversarial Networks (GAN)
    • Monday: Generative Adversarial Networks (GAN); Optional related reading: Original GAN paper, Deep Convolutional GANs
    • Wednesday: GANs (continued, updated slides)
    • Friday: Deep Convolutional GAN (DCGAN); DCGAN MNIST tutorial (notebook, pdf); Optional: Transposed convolution visualization (notebook, pdf
11. Week 11 (11/1/2021) - Generative Adversarial Networks (GAN)
    • Monday: DCGAN (continued)
    • Wednesday: Wasserstein GAN; Optional related reading: Original Wasserstein GAN paper, Improved WGAN paper with gradient penalty, A Primer on Optimal Transport
    • Friday: Wasserstein GAN (continued)
12. Week 12 (11/8/2021) - Normalizing Flows
    • Monday: Normalizing Flows; Change of variables demo (notebook, pdf)
    • Wednesday: Normalizing Flows (continued)
    • Friday: Normalizing Flows (continued)
13. Week 13 (11/15/2021)
    • Monday: Iterative flows via density destructors
    • Wednesday: Iterative flows and iterative alignment (updated slides, continued)
    • Friday: Topic models
14. Week 14 (11/22/2021)
    • Monday: Word embeddings (Word2Vec)