With the earlier-than-expected release of Course 313. Advanced Neural Network Methods , it's time to update my course creation plans again. We have a fascinating lineup.
Course 314. Neural Network Optimization
March 1, 2020
In this course we'll focus on tuning our neural network up for optimal performance in a practical application - compressing imagery from the Mars Rover for efficient transmission back to Earth. We'll demonstrate the importance of having a well-defined goal and performance measurement. We'll also show how to use profiling to find slow spots in our code, and how to widen out those bottlenecks without resorting to GPUs. (All the code is sized to run on your local machine.)
At the core of this course is hyperparameter optimization, a non-convex optimization problem in a hybrid continuous-discrete parameter space (read as "hard"). Since we can't rely on gradient descent for this, we'll explore other options and show their benefits and weaknesses on our Martian imagery compression problem. By the time we're done, you'll have a solid grasp of a rich hyperparameter optimization toolkit including grid search, random search, and Powell's method.
Course 315. Convolutional Neural Networks
June 1, 2020
An overwhelmingly popular application of deep neural networks is image classification. We'll use a pixel-level classification task this to build convolutional layers, pooling, and softmax layers into our framework and show how they work. We'll also get to demonstrate the ways CNNs can be fooled and power of of feature engineering.
Course 316. Recurrent Neural Networks
September 1, 2020
The recent surge in popularity of GPT-2 and Sesame-Street-themed natural language processing models have thrown recurrent neural networks into the spotlight. In this course, we'll get to dig deep into what makes them special, what makes them tricky to work with, and how to reduce them to code. As always, we'll build them from scratch using a practical problem to anchor us and demonstrate our work.
Please keep in mind that these are my best guesses at how things will play out, but the course creation process is more like hiking cross country than driving with GPS. It's a series of surprising vistas, dead-ends, and side-tracks that lure the traveler deep into the woods and can be resisted only with great effort. The dates, and possibly even the course sequence, will probably change a bit.