# dliss-tutorial
Tutorial for [International Summer School
on Deep Learning, 2019](http://dl-lab.eu/) in Gdansk, Poland

## Sections

### Overview Talk

https://docs.google.com/presentation/d/1DJI1yX4U5IgApGwavt0AmOCLWwso7ou1Un93sMuAWmA/

### Tutorial
There are currently 3 hands-on sections to this tutorial.

- The [first section](1_pretrained_vectors.ipynb) covers pre-trained word embeddings [(colab)](https://colab.research.google.com/github/dpressel/dlss-tutorial/blob/master/1_pretrained_vectors.ipynb)

- The [second section](2_context_vectors.ipynb) covers pre-trained contextual emeddings [(colab)](https://colab.research.google.com/github/dpressel/dlss-tutorial/blob/master/2_context_vectors.ipynb)
- The [third section](3_finetuning.ipynb) covers fine-tuning a pre-trained model [(colab)](https://colab.research.google.com/github/dpressel/dlss-tutorial/blob/master/3_finetuning.ipynb)

### Updates

- *April 2022* If you are interested in learning how to build different Transformer architectures from the ground up, I have a [new set of tutorials](https://github.com/dpressel/tfs) with in-depth details and full implementations of several popular Transformer models.  They show how to build models step by step, how to pretrain them, and how to use them for downstream tasks.  There is an accompanying Python package that contains all of the tutorial pieces put together



- *July 2020* I have posted a set of [Colab tutorials](https://github.com/dpressel/mead-tutorials) using [MEAD](https://github.com/dpressel/mead-baseline) which is referenced in these tutorials.  This new set of notebooks covers similar material, including transfer learning for classification and taggers, as well as training Transformer-based models from scratch using the [MEAD API](https://github.com/dpressel/mead-baseline/tree/master/layers) with TPUs.  MEAD makes it easy to train lots of powerful models for NLP using a simple YAML configuration and makes it easy to extend the code with new models while comparing against strong baselines!
