## Materials
* [__slides__](https://github.com/yandexdataschool/Practical_RL/files/15286337/ysda_practical_rl_lecture_04.pdf)
* Our [lecture](https://yadi.sk/i/Gd9yWV1dpuB7BQ), [seminar](https://yadi.sk/i/mvtKAIRN2yKU2g) (russian)
* [__seminar slides__](https://yadi.sk/i/B89gXClSpmYZKw)


* David Silver lecture - [video](https://www.youtube.com/watch?v=UoPei5o4fps)
* More practical and less theoretical lecture from MIT 6.S191 - [video](https://www.youtube.com/watch?v=xWe58WGWmlk)
* Understanding approximate q-learning - [url](https://danieltakeshi.github.io/2016/10/31/going-deeper-into-reinforcement-learning-understanding-q-learning-and-linear-function-approximation/)
* Karpathy's post on approximate RL - [url](http://karpathy.github.io/2016/05/31/rl/)

## More materials
* __[recommended]__ How to _actually_ do deep reinforcement learning by J. Schulman - [pdf](http://rll.berkeley.edu/deeprlcourse/docs/nuts-and-bolts.pdf)
* __[recommended]__ [Spinning up](https://spinningup.openai.com/en/latest/) - a massive repository of deep RL knowledge
* __[recommended]__ A table of collateral effects of reinforcement learning in games - [table](https://docs.google.com/spreadsheets/d/e/2PACX-1vRPiprOaC3HsCf5Tuum8bRfzYUiKLRqJmbOoC-32JorNdfyTiRRsR7Ea5eWtvsWzuxo8bjOxCG84dAg/pubhtml?fbclid=IwAR114qpVxlzU8vO8BiMRbpPKUIci8sHsOoUZ16-_0yVtABr6AW1-AZKFXWg)
* DQN and modiffications - lecture by J. Schulman - [video](https://www.youtube.com/watch?v=h1-pj4Y9-kM)
* * interactive demos in your browser: [demo1](http://cs.stanford.edu/people/karpathy/convnetjs/demo/rldemo.html)(karpathy), [demo2](http://janhuenermann.com/projects/learning-to-drive)(Hünermann)
* Reinforcement learning architectures list - [repo](https://github.com/5vision/deep-reinforcement-learning-networks)
* Article on dueling DQN - [arxiv](https://arxiv.org/pdf/1511.06581.pdf)
* Article on double DQN - [arxiv](https://arxiv.org/abs/1509.06461)
* Article on prioritized experience replay - [arxiv](https://arxiv.org/abs/1511.05952)
* Article on Rainbow: Combining Improvements in Deep Reinforcement Learning - [arxiv](https://arxiv.org/abs/1710.02298)
* Article on bootstrap DQN - [pdf](https://papers.nips.cc/paper/6501-deep-exploration-via-bootstrapped-dqn.pdf), [summary](http://pemami4911.github.io/paper-summaries/2016/08/16/Deep-exploration.html)
* Article on asynchronuous methods in deep RL - [arxiv](https://arxiv.org/abs/1602.01783)
* Successor representations for reinforcement learning - [article](https://arxiv.org/abs/1606.02396), [video](https://www.youtube.com/watch?v=kNqXCn7K-BM&feature=youtu.be)
* Video on asynchronuous methods (Mnih) - [video](https://www.youtube.com/watch?v=9sx1_u2qVhQ)
* An overview of deep reinforcement learning - [arxiv](https://arxiv.org/pdf/1701.07274v1.pdf)

## DQN tutorials
* [in pytorch] A great series starting from simple DQN to all the cool new stuff - [url](https://github.com/higgsfield/RL-Adventure)
* A guide to deep RL from ~scratch (nervana blog) - [url](https://www.nervanasys.com/demystifying-deep-reinforcement-learning/)
* Building deep q-network from ~scratch (blog) - [url](https://jaromiru.com/2016/09/27/lets-make-a-dqn-theory/)
* Another guide guide to DQN from ~scratch (blog) - [url](https://rubenfiszel.github.io/posts/rl4j/2016-08-24-Reinforcement-Learning-and-DQN.html)


## Practice

* Seminar: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/master/week04_approx_rl/seminar_pytorch.ipynb)
* Homework (main): [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/master/week04_approx_rl/homework_pytorch_main.ipynb#scrollTo=KVvvo7k_ap8w)
* Homework (debug): [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/master/week04_approx_rl/homework_pytorch_debug.ipynb#scrollTo=KVvvo7k_ap8w)



From now on, we have two tracks, for pytorch and tensorflow. However, pytorch track is somewhat better supported by the course team. You can choose whichever track you want, but unless you're expertly familiar with your framework, we recommend you to start by completing the task in pytorch and only then reproduce your solution in your chosen framework.

Begin with `seminar_<framework>.ipynb` and then proceed with `homework_<framework>.ipynb`.


__Note: you're not required to submit assignments in all three frameworks. Pick one and go with it. Maybe switch it occasionally if you want more challenge.__
