Updated material on NNX

Author: jlperla

README.md

@@ -77,30 +77,30 @@ Slides for the lectures can be found [here](https://ubcecon.github.io/ECON622/le
     - **SLIDES**: [Factorizations and Direct Methods](https://ubcecon.github.io/ECON622/lectures/lectures/factorizations_direct_methods.html)
     - Introduction to regularization and implicit bias of algorithms
     - [Numerical Linear Algebra](https://julia.quantecon.org/tools_and_techniques/numerical_linear_algebra.html) applying generic programming
-13.  **October 23**: Iterative Methods, Geometry of Optimization, and Rethinking LLS
+13.  **October 23**: Iterative Methods, Geometry of Optimization, Rethinking LLS, and Preconditioning
     - **SLIDES**: [Least Squares](https://ubcecon.github.io/ECON622/lectures/lectures/least_squares.html) and [Iterative Methods](https://ubcecon.github.io/ECON622/lectures/lectures/iterative_methods.html)
     - [Iterative Methods](https://julia.quantecon.org/tools_and_techniques/iterative_methods_sparsity.html)
-14. **October 28**: Preconditioning and Overview of Machine Learning
-    - **SLIDES**:  [Iterative Methods](https://ubcecon.github.io/ECON622/lectures/lectures/iterative_methods.html) and [Intro to ML](https://ubcecon.github.io/ECON622/lectures/lectures/intro_to_ml.html)
+14. **October 28**: Overview of Machine Learning
+    - **SLIDES**:  [Intro to ML](https://ubcecon.github.io/ECON622/lectures/lectures/intro_to_ml.html)
     - Finalize discussion of iterative methods and preconditioning
     - Introduce key concepts about supervised, unsupervised, reinforcement learning, semi-supervised, kernel-methods, deep-learning, etc.
+    - Basic introduction to JAX and Python frameworks
 15. **October 30**: Differentiable everything! JAX and Auto-Differentiation/JVP/etc.
-    - **SLIDES**: Finish [Intro to ML](https://ubcecon.github.io/ECON622/lectures/lectures/intro_to_ml.html) and start [Differentiation](https://ubcecon.github.io/ECON622/lectures/lectures/differentiation.html)
-    - Core JAX transforms
+    - **SLIDES**: [Differentiation](https://ubcecon.github.io/ECON622/lectures/lectures/differentiation.html)
     - Reverse-mode and forward-mode AD.
     - Jvps and vjps
-16. **November 4**: AD of Implicit Functions, Optimization Methods for High-Dimensional Problems
-    - **SLIDES**: Finish [Differentiation](https://ubcecon.github.io/ECON622/lectures/lectures/differentiation.html) and start [Optimization](https://ubcecon.github.io/ECON622/lectures/lectures/optimization.html)
     - Implicit differentiation of systems of ODEs, linear systems, etc.
+16. **November 4**: High-dimensional optimization and Stochastic Optimization
+    - **SLIDES**: [Optimization](https://ubcecon.github.io/ECON622/lectures/lectures/optimization.html)
     - Gradient descent variations
-17. **November 6**: Stochastic Optimization Methods
+    - Using unbiased estimates instead of gradients
+17. **November 6**: Stochastic Optimization Methods and Machine Learning Pipelines
     - **SLIDES**: SGD variations in [Optimization](https://ubcecon.github.io/ECON622/lectures/lectures/optimization.html)
+    - W&B sweeps, and code in `lectures/lectures/examples`- 
     - SGD and methods for variance reduction in gradient estimates
     - Using SGD-variants in practice within ML pipelines in JAX and Pytorch
-18. **November 18**: Machine Learning Pipelines, HPO, and ERM
-    - **SLIDES**: Finished example code of pipelines in [Optimization](https://ubcecon.github.io/ECON622/lectures/lectures/optimization.html), W&B sweeps, and code in `lectures/lectures/examples`
     - **Readings**: [Probabilistic Machine Learning: An Introduction](https://probml.github.io/pml-book/book1.html) Section 5.4 on ERM
-19. **November 20**: Neural Networks, Representation Learning, Double-Descent
+18. **November 18**: Neural Networks, Representation Learning, Double-Descent
     - **SLIDES**: [Deep Learning and Representation Learning](https://ubcecon.github.io/ECON622/lectures/lectures/deep_learning.html) and started [Double-Descent and Regularization](https://ubcecon.github.io/ECON622/lectures/lectures/overparameterization.html)
     - **Readings**
       - [Probabilistic Machine Learning: An Introduction](https://probml.github.io/pml-book/book1.html) Section 13.2.1 to 13.2.6 on MLPs and the importance of depth
@@ -109,7 +109,7 @@ Slides for the lectures can be found [here](https://ubcecon.github.io/ECON622/le
       - [Mark Schmidt's CPSC440 Notes on Double-Descent Curves](https://www.cs.ubc.ca/~schmidtm/Courses/440-W22/L7.pdf) (see [CPSC340](https://www.cs.ubc.ca/~schmidtm/Courses/340-F22/L32.pdf) lectures for a more basic treatment of these topics)
     - **Optional Extra Material**
       - [Probabilistic Machine Learning: Advanced Topics](https://probml.github.io/pml-book/book2.html) Section 32 on representation learning
-20. **November 25** Finish Double-Descent and Intro to Kernel Methods and Gaussian Processes
+19. **November 20** Finish Double-Descent and Intro to Kernel Methods and Gaussian Processes
     - **SLIDES**: [Kernel Methods and Gaussian Processes](https://ubcecon.github.io/ECON622/lectures/lectures/kernel_methods.html) and finish [Double-Descent and Regularization](https://ubcecon.github.io/ECON622/lectures/lectures/overparameterization.html)
     - **Readings**
       - If you didn't do it already, read [Mark Schmidt's CPSC440 Notes on Double-Descent Curves and Overparameterization](https://www.cs.ubc.ca/~schmidtm/Courses/440-W22/L7.pdf) (see [CPSC340](https://www.cs.ubc.ca/~schmidtm/Courses/340-F22/L32.pdf) lectures for a more basic treatment of these topics)
@@ -119,13 +119,12 @@ Slides for the lectures can be found [here](https://ubcecon.github.io/ECON622/le
       -  [Probabilistic Machine Learning: Advanced Topics](https://probml.github.io/pml-book/book2.html) Section 18.1 to 18.3 on GPs and kernels
       - Researchers working in GPs love the online textbook [Gaussian Processes for Machine Learning](https://gaussianprocess.org/gpml/chapters/), so you may want to read the intro section on [GP Regression](https://gaussianprocess.org/gpml/chapters/RW2.pdf)
 
+20. **November 25** Bayesian Methods and HMC
 21. **November 27** Applications
 22. **December 2** Applications
-
-
 23. **December 4** Applications
 
-28. **December 18**
+24. **December 18**
     - Final Project due
 
 

docs/lectures/lectures/deep_learning.ipynb

@@ -628,7 +628,7 @@
         "    -   Good representations are much more robust\n",
         "    -   With transfer learning you may be able to adapt very easily"
       ],
-      "id": "7cd249fa-049d-40b3-b144-041bc5b140a8"
+      "id": "1559a0b9-88e7-455d-bfab-f67992541006"
     }
   ],
   "nbformat": 4,