## Assignments

Just as usual, start with  `seminar_MCTS.ipynb`
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/master/week10_planning/seminar_MCTS.ipynb)

## Materials: planning

* Planning by dynamic programming (D. Silver) - [video](https://www.youtube.com/watch?v=Nd1-UUMVfz4)
* Planning via tree search [videos 2-6 from CS188](https://www.youtube.com/channel/UCHBzJsIcRIVuzzHVYabikTQ)
* Our lecture:
  * [Lecture(planning, MCTS, planning in POMDP)](https://yadi.sk/i/lOAUu7o13JBHFz) & [Seminar(MCTS)](https://yadi.sk/i/bkmjEZrk3JBHGF)
  * Slides [part1](https://yadi.sk/i/3PM9zCP33J3ub3) (intro), [part2](https://yadi.sk/i/M03xvZ2y3JMQre) (pomdp)
* Monte-carlo tree search
  * Monte-carlo tree search step-by-step by J.Levine (very intuitive) - [video](https://www.youtube.com/watch?v=UXW2yZndl7U)
  * Udacity video on monte-carlo tree search (first part of a chain) - [video](https://www.youtube.com/watch?v=onBYsen2_eA)
  * A Survey of Monte Carlo Tree Search Methods (2011-2012) [pdf](http://ccg.doc.gold.ac.uk/ccg_old/papers/browne_tciaig12_1.pdf)
  * Reminder: UCB-1 - [slides](https://www.cs.bham.ac.uk/internal/courses/robotics/lectures/ucb1.pdf)
  * Guide to MCTS - [url](https://jeffbradberry.com/posts/2015/09/intro-to-monte-carlo-tree-search/)
  or [cached](https://webcache.googleusercontent.com/search?q=cache:jeffbradberry.com/posts/2015/09/intro-to-monte-carlo-tree-search/)

## Supplementary materials

* Integrating learning and planning (D. Silver) - [video](https://www.youtube.com/watch?v=ItMutbeOHtc)
* Approximating the MCTS optimal actions - 5vision solution for deephack.RL, code by Mikhail Pavlov - [repo](https://github.com/5vision/uct_atari)
* Alpha Go - [explanation](https://medium.com/@jonathan_hui/alphago-how-it-works-technically-26ddcc085319)
* Monte Carlo Planning in POMDP [pdf](https://papers.nips.cc/paper/4031-monte-carlo-planning-in-large-pomdps.pdf)
