Practice with PyTorch and Dataloaders:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/girafe-ai/ml-course/blob/22f_basic/week0_08_dropout_batchnorm/practice_pytorch_and_dataloaders.ipynb)

Vanishing gradient example:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/girafe-ai/ml-course/blob/22f_basic/week0_08_dropout_batchnorm/vanishing_grad_example.ipynb)


__Further readings__:
* [en] Notes on vector and matrix derivatives: http://cs231n.stanford.edu/vecDerivs.pdf
* [en] Stanford notes on backpropagation: http://cs231n.github.io/optimization-2/
* [en] Stanford notes on different activation functions (and just intuition): http://cs231n.github.io/neural-networks-1/
* [en] Great post on Medium by Andrej Karpathy: https://medium.com/@karpathy/yes-you-should-understand-backprop-e2f06eab496b
* [en] CS231n notes on data preparation (batch normalization over there): http://cs231n.github.io/neural-networks-2/
* [en] CS231n notes on gradient methods: http://cs231n.github.io/neural-networks-3/
* [en] Original paper introducing Batch Normalization: https://arxiv.org/pdf/1502.03167.pdf
* [en] What Every Computer Scientist Should Know About Floating-Point Arithmetic: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
