CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 1
Homework 1: Autoregressive Models
Deliverable: This PDF write-up by Tuesday February 11th, 23:59pm. Your PDF should be generated by simply replacing the placeholder images of this LaTeX document with the appropriate solution images that will be generated automatically when solving each question. The solution images are automatically generated and saved using the accompanying IPython notebook. Your PDF is to be submitted into Gradescope. This PDF already contains a few solution images. These images will allow you to check your own solution to ensure correctness.
Question 1: 1D Data
(a) [10pt] Fitting a Histogram
Final test loss for dataset 1: 2.0553 nats / dim
(a) Dataset 1: Training curve
Final test loss for dataset 2: 0.0000 nats / dim
(a) Dataset 2: Training curve
(b) Dataset 1: Learned distribution
(b) Dataset 2: Learned distribution
CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 2 (b) [10pth] Fitting Discretized Mixture of Logistics
Final test loss for dataset 1: 2.0586 nats / dim
(a) Dataset 1: Training curve (b) Dataset 1: Learned distribution
Final test loss for dataset 2: 0.0000 nats / dim
(a) Dataset 2: Training curve (b) Dataset 2: Learned distribution
CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 3 Question 2: MADE
(a) [10pt] Fitting 2D Data
Final test loss for dataset 1: 3.1518 nats / dim
(a) Dataset 1: Training curve
Final test loss for dataset 2: 0.0000 nats / dim
(a) Dataset 2: Training curve
(b) Dataset 1: Learned distribution
(b) Dataset 2: Learned distribution
CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 4 (b) [10pt] Shapes and MNIST
Final test loss for dataset 1: 0.0623 nats / dim
(a) Dataset 1: Training curve (b) Dataset 1: Samples
Final test loss for dataset 2: 0.0000 nats / dim
(a) Dataset 2: Training curve (b) Dataset 2: Samples
CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 5
Question 3: PixelCNNs
(a) [15pt] PixelCNNs on Shapes and MNIST
Final test loss for dataset 1: 0.0420 nats / dim
(a) Dataset 1: Training curve
Final test loss for dataset 2: 0.0000 nats / dim
(a) Dataset 2: Training curve
(b) Dataset 1: Samples
(b) Dataset 2: Samples
CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 6 (b) [15pt] PixelCNN on Colored Shapes and MNIST: Independent Color Channels
Final test loss for dataset 1: 0.0444 nats / dim
(a) Dataset 1: Training curve (b) Dataset 1: Samples
Final test loss for dataset 2: 0.0000 nats / dim
(a) Dataset 2: Training curve (b) Dataset 2: Samples
CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 7 (c) [15pt] PixelCNN on Colored Shapes and MNIST: Autoregressive Color Channels
Final test loss for dataset 1: 0.0236 nats / dim
(a) Dataset 1: Training curve (b) Dataset 1: Samples
Final test loss for dataset 2: 0.0000 nats / dim
(a) Dataset 2: Training curve (b) Dataset 2: Samples
CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 8 (d) [15pt] Conditional PixelCNNs
Final test loss for dataset 1: 0.0368 nats / dim
(a) Dataset 1: Training curve (b) Dataset 1: Samples
Final test loss for dataset 2: 0.0000 nats / dim
(a) Dataset 2: Training curve (b) Dataset 2: Samples
CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 9
Bonus Questions (Optional) 1. [10pt] Gated PixelCNN
Final test loss: 0.0000 nats / dim
(a) Training curve (b) Samples
CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 10 2. [10pt] Grayscale PixelCNN
Final test loss: 0.0000 nats / dim
(a) Training curve (b) Samples
CS 294-158 Deep Unsupervised Learning, Homework 1, Spring 2020 11 3. [10pt] Parallel Multiscale PixelCNN
Final test loss: 0.0000 nats / dim
(a) Training curve (b) Samples