{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mrm8488/shared_colab_notebooks/blob/master/T5_qgsquad_with_HF_transformers_for_QG.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RXr_ss_50qPf"
      },
      "source": [
        "# Tutorial to fine-tune Google's text2text/seq2seq T5 model on SQuAD v1 dataset for AQG (Automatic Question Generation) 🤖 ✍ ❓\n",
        "\n",
        "Author: **Manuel Romero** [mrm8488](https://twitter.com/mrm8488). NLP/NLG Senior Engineer at [Narrativa](https://narrativa.com/) and Head collaborator at [HuggingFace](https://huggingface.co/mrm8488)\n",
        "\n",
        "> Note: This is still WIP so you may find errors"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GdiqFsx0VLBY"
      },
      "source": [
        "## The model\n",
        "## [Exploring Transfer Learning with **T5**: the Text-To-Text Transfer Transformer](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "omPVRYUmUxup"
      },
      "source": [
        "![T5.png](data:image/png;base64,[encoded data, 219980 characters])\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ae9yi7fiVl6b"
      },
      "source": [
        "## The dataset\n",
        "## [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/explore/1.1/dev/)\n",
        "### Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VZR8jUn_BwzU"
      },
      "source": [
        "## The procedure\n",
        "\n",
        "### There are several ways to train models for Question Generation. In this tutorial, we will prepend the `answer` to the `context`send it through the enconder and then the decoder will have to figure out the `question`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4EDo89yfWVEn"
      },
      "source": [
        "## Training with 🤗 ecosystem"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "21hGe7Mr0gos",
        "outputId": "3f6080c9-2d19-42b2-becd-c8d81d0d05c8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue May 24 15:32:09 2022       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla V100-SXM2...  Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   36C    P0    27W / 300W |      2MiB / 16160MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "! nvidia-smi"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "28K5CDBE08jh"
      },
      "outputs": [],
      "source": [
        "! pip install -q transformers datasets rouge_score"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "agUs01Ms12b_"
      },
      "source": [
        "## HF auth"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KE_RTo78sqb-",
        "outputId": "0f6d4bab-9d63-4d8a-ddff-9de71b2d2ffd"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "        _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|\n",
            "        _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|\n",
            "        _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|\n",
            "        _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|\n",
            "        _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|\n",
            "\n",
            "        To login, `huggingface_hub` now requires a token generated from https://huggingface.co/settings/tokens.\n",
            "        (Deprecated, will be removed in v0.3.0) To login with username and password instead, interrupt with Ctrl+C.\n",
            "        \n",
            "Token: \n",
            "Login successful\n",
            "Your token has been saved to /root/.huggingface/token\n",
            "\u001b[1m\u001b[31mAuthenticated through git-credential store but this isn't the helper defined on your machine.\n",
            "You might have to re-authenticate when pushing to the Hugging Face Hub. Run the following command in your terminal in case you want to set this credential helper as the default\n",
            "\n",
            "git config --global credential.helper store\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "!huggingface-cli login"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "IADibkaEs79C"
      },
      "outputs": [],
      "source": [
        "! git config --global credential.helper store"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qZQ0lpecFdiF"
      },
      "source": [
        "## Model and tokenizer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XBTV-c1nFiAX",
        "outputId": "d5294f32-56e1-495a-80c7-5a95273459b5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/transformers/models/t5/tokenization_t5_fast.py:161: FutureWarning: This tokenizer was incorrectly instantiated with a model max length of 512 which will be corrected in Transformers v5.\n",
            "For now, this behavior is kept to avoid breaking backwards compatibility when padding/encoding with `truncation is True`.\n",
            "- Be aware that you SHOULD NOT rely on t5-small automatically truncating your input to 512 when padding/encoding.\n",
            "- If you want to encode/pad to sequences longer than 512 you can either instantiate this tokenizer with `model_max_length` or pass `max_length` when encoding/padding.\n",
            "- To avoid this warning, please instantiate this tokenizer with `model_max_length` set to your preferred value.\n",
            "  FutureWarning,\n"
          ]
        }
      ],
      "source": [
        "CKPT = 't5-small'\n",
        "from transformers import AutoTokenizer, T5ForConditionalGeneration\n",
        "tokenizer = AutoTokenizer.from_pretrained(CKPT)\n",
        "model = T5ForConditionalGeneration.from_pretrained(CKPT)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OcC0F2ED2CKJ"
      },
      "source": [
        "## Dataset preparation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1UxMVyz5294I"
      },
      "source": [
        "### Let's load the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 84,
          "referenced_widgets": [
            "634eb7c5672d4e1fa626d5bb1ac2c82b",
            "30e4b4385f164a1b9a750b4203b1ffb1",
            "d30b3558c4244fa88ee1a560bd0c5a7b",
            "64e7ac4067654e79bd4a4dd39a93f13c",
            "bf9a816ff31a4a3883b4c8550ce1820c",
            "5830e80a06cb43dd85a9bb3d941e3737",
            "7d8c4f73a8784d04a7b641e0639a4e1b",
            "26b3ef6b48f64746b806d112b54a06c3",
            "20f6a8a53daa4c109a3a6064bb4f6466",
            "8eb4fb77df6a4388ae4a7d239de9cf0c",
            "631cb403f11348dfafd333b9acc9237b"
          ]
        },
        "id": "hM3s5q981Ga6",
        "outputId": "fd46966c-521c-4bb3-85a0-4aeeafbfe3ce"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Using custom data configuration default\n",
            "Reusing dataset qg_squad (/root/.cache/huggingface/datasets/asahi417___qg_squad/default/0.0.0/6e170fd3cee937658059107ad53d4b6da51416c7c3e2cddbb3c8ab57b4fd1bd5)\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/3 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "634eb7c5672d4e1fa626d5bb1ac2c82b"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "from datasets import load_dataset, load_metric\n",
        "\n",
        "dataset = load_dataset('asahi417/qg_squad')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZewvnLHJ3CQX"
      },
      "source": [
        "### Let's explore the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4b8roCmG1tTA",
        "outputId": "8ef925c2-6310-4ded-c1c1-7dedeb988111"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    train: Dataset({\n",
              "        features: ['answer', 'question', 'sentence', 'paragraph', 'sentence_answer', 'paragraph_answer', 'paragraph_sentence'],\n",
              "        num_rows: 75722\n",
              "    })\n",
              "    validation: Dataset({\n",
              "        features: ['answer', 'question', 'sentence', 'paragraph', 'sentence_answer', 'paragraph_answer', 'paragraph_sentence'],\n",
              "        num_rows: 10570\n",
              "    })\n",
              "    test: Dataset({\n",
              "        features: ['answer', 'question', 'sentence', 'paragraph', 'sentence_answer', 'paragraph_answer', 'paragraph_sentence'],\n",
              "        num_rows: 11877\n",
              "    })\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_5V6i4yG2omO",
        "outputId": "8b3df43e-160b-4bf3-d8fa-c6f830013768"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'answer': 'established beliefs or customs',\n",
              " 'paragraph': \"Heresy is any provocative belief or theory that is strongly at variance with established beliefs or customs. A heretic is a proponent of such claims or beliefs. Heresy is distinct from both apostasy, which is the explicit renunciation of one's religion, principles or cause, and blasphemy, which is an impious utterance or action concerning God or sacred things.\",\n",
              " 'paragraph_answer': \"Heresy is any provocative belief or theory that is strongly at variance with <hl> established beliefs or customs <hl>. A heretic is a proponent of such claims or beliefs. Heresy is distinct from both apostasy, which is the explicit renunciation of one's religion, principles or cause, and blasphemy, which is an impious utterance or action concerning God or sacred things.\",\n",
              " 'paragraph_sentence': \"<hl> Heresy is any provocative belief or theory that is strongly at variance with established beliefs or customs . <hl> A heretic is a proponent of such claims or beliefs. Heresy is distinct from both apostasy, which is the explicit renunciation of one's religion, principles or cause, and blasphemy, which is an impious utterance or action concerning God or sacred things.\",\n",
              " 'question': 'What is heresy mainly at odds with?',\n",
              " 'sentence': 'Heresy is any provocative belief or theory that is strongly at variance with established beliefs or customs .',\n",
              " 'sentence_answer': 'Heresy is any provocative belief or theory that is strongly at variance with <hl> established beliefs or customs <hl> .'}"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "dataset['train'][0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x7L8T_VN3LYg"
      },
      "source": [
        "### Let's format the dataset\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "gaTsiEd0D-hU"
      },
      "outputs": [],
      "source": [
        "def format_dataset(example):\n",
        " return {'input': 'answer: ' + example['answer'] + ' context: ' + example['paragraph'], 'target': example['question']}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mt-c0E0GEPI7",
        "outputId": "a7c88e7c-af35-4c1f-dae0-b6686c26a47a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Loading cached processed dataset at /root/.cache/huggingface/datasets/asahi417___qg_squad/default/0.0.0/6e170fd3cee937658059107ad53d4b6da51416c7c3e2cddbb3c8ab57b4fd1bd5/cache-046d9105525c961b.arrow\n",
            "Loading cached processed dataset at /root/.cache/huggingface/datasets/asahi417___qg_squad/default/0.0.0/6e170fd3cee937658059107ad53d4b6da51416c7c3e2cddbb3c8ab57b4fd1bd5/cache-eeab071e548283b1.arrow\n",
            "Loading cached processed dataset at /root/.cache/huggingface/datasets/asahi417___qg_squad/default/0.0.0/6e170fd3cee937658059107ad53d4b6da51416c7c3e2cddbb3c8ab57b4fd1bd5/cache-04f584d3f35c1dc4.arrow\n"
          ]
        }
      ],
      "source": [
        "dataset = dataset.map(format_dataset, remove_columns=dataset['train'].column_names)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bgWNGEJPFIdA",
        "outputId": "79717bde-c6f9-4622-e20a-a90d24e342ee"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['input', 'target'],\n",
              "    num_rows: 75722\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "dataset['train']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Apw09O1-M07K"
      },
      "source": [
        "### Let's calculate the inputs and targets lengths"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 266,
          "referenced_widgets": [
            "8059c03298d1444ea524d7288d9fafa2",
            "c55b9db0b4c84e0b8bc6592f620435f2",
            "cd488407d9564790aae16ac2664ad89e",
            "47029952f00a4d7a9b53166913612273",
            "2a1dd1f17eb048828e159d36a80a84fb",
            "30e1d632b0ad486888286b2f4474e222",
            "1214d750cd524ce086258ca9689a91e1",
            "64b08505aad3482db93c06b93d41a8f0",
            "29bf66f806c04c58b5d2c20896eaad50",
            "d7b27900fa89460980bee7bd66ba7c69",
            "1d6476c677264238843fde427d464496",
            "cad4716558644f788f6afd5b83b37a3d",
            "28e910bd55cb41459f8a5b5779cacfff",
            "ae8af3448e944352862cd22e723df86d",
            "74cc87c09ee04d5096688d8b54b547da",
            "abf7378bff344d6e91ea7fa2d193e735",
            "2b45838756d94c0c910ecc8c95badcf4",
            "145ee2480bac4ad0b5e015c9e6987433",
            "888d6d377396443dafeb941d6badb025",
            "0bba5c87272742f59767e58e14711351",
            "1be13c9fe339499aa7a837153b613cb5",
            "30536dc4f57749db91e890ebbb4c1781",
            "6092d2342791482da207f191f80a31e9",
            "3c6a218e5a2f4d0fbe371d5685b7f144",
            "68eece9473c449d09e8df3de5bd83591",
            "b5b079ca962e4e1689426c4d05992f9b",
            "3162fb09a87c49d6bc415d346d3f27ae",
            "67aa5863e89b436e9bbffd46396a92b9",
            "b1fb7b59a746453db258ea76fe9bbf0f",
            "c28f26eafae541b6924d3cce9d910ad2",
            "6c5cbb433b4646e88259b76cffc65925",
            "df709b3471c64d649e13ef03af161f8d",
            "25dfe1f2501d4bcb9d67b7b09d26a73a",
            "5d89e1e341714c5fabd3b5452b82bcb6",
            "a0649663e0064c3898698e6ac70f8f0b",
            "e7fcf56d72ae4db2b58184bd3cce77bf",
            "49d08e1666744788935d5cec78debec5",
            "0b0c3bfa408944f0ab53ceacbb0290ab",
            "c65b1168a3844f3094282558ad6daec4",
            "9f573622dba7461c80befb6147e821ed",
            "942e27107b664be995ccaac6938b395f",
            "f11c5e703b8a41998f9f2e904d91ab97",
            "bfc4c27526ea463995d5ef9d25d8f87f",
            "8eac35e6f76849de84986bd3888b9274",
            "d5b87682b53442bd8659eb783c7014e6",
            "0a4beea1de764511995885126c16dcfe",
            "024a4cc1b48944618993bdea6b4985b7",
            "35f1a260b5b14a8eb1f22a590f681b55",
            "8d904142c55740179553116f7035ae73",
            "cae17ca9e5fe42ab8939059cb532d961",
            "28893a1b202b492b9dda6374c7bd166f",
            "518917b0fec24c248d26f8dfd8f1eafd",
            "826159a8ace64488830020c7adccfa9f",
            "39c9cd1be375485e8f0ee7928485c715",
            "7f047cb9d0e344bf9d4935fdc65fda3e"
          ]
        },
        "id": "LT7ovuoJFLFC",
        "outputId": "917a5fdc-2a3a-4d65-9fd7-72d32494f0bb"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "        "
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "#0:   0%|          | 0/2500 [00:00<?, ?ex/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "8059c03298d1444ea524d7288d9fafa2"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "#3:   0%|          | 0/2500 [00:00<?, ?ex/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "cad4716558644f788f6afd5b83b37a3d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "#1:   0%|          | 0/2500 [00:00<?, ?ex/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "6092d2342791482da207f191f80a31e9"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "#2:   0%|          | 0/2500 [00:00<?, ?ex/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "5d89e1e341714c5fabd3b5452b82bcb6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Token indices sequence length is longer than the specified maximum sequence length for this model (515 > 512). Running this sequence through the model will result in indexing errors\n",
            "Token indices sequence length is longer than the specified maximum sequence length for this model (542 > 512). Running this sequence through the model will result in indexing errors\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/1 [00:00<?, ?ba/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d5b87682b53442bd8659eb783c7014e6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Input Mean: 141.1058, %-Input > 512:0.0015,  %-Input > 256:0.0735, %-Input > 128:0.4831 Output Mean:14.9089, %-Output > 128:0.0, %-Output > 64:0.0, %-Output > 16:0.3018\n"
          ]
        }
      ],
      "source": [
        "def map_to_length(x):\n",
        "  x[\"input_len\"] = len(tokenizer(x[\"input\"]).input_ids)\n",
        "  x[\"input_longer_512\"] = int(x[\"input_len\"] > 512)\n",
        "  x[\"input_longer_256\"] = int(x[\"input_len\"] > 256)\n",
        "  x[\"input_longer_128\"] = int(x[\"input_len\"] > 128)\n",
        "  x[\"out_len\"] = len(tokenizer(x[\"target\"]).input_ids)\n",
        "  x[\"out_longer_128\"] = int(x[\"out_len\"] > 128)\n",
        "  x[\"out_longer_64\"] = int(x[\"out_len\"] > 64)\n",
        "  x[\"out_longer_16\"] = int(x[\"out_len\"] > 16)\n",
        "  return x\n",
        "\n",
        "sample_size = 10000\n",
        "data_stats = dataset['train'].select(range(sample_size)).map(map_to_length, num_proc=4)\n",
        "\n",
        "def compute_and_print_stats(x):\n",
        "  if len(x[\"input_len\"]) == sample_size:\n",
        "    print(\n",
        "        \"Input Mean: {}, %-Input > 512:{},  %-Input > 256:{}, %-Input > 128:{} Output Mean:{}, %-Output > 128:{}, %-Output > 64:{}, %-Output > 16:{}\".format(\n",
        "            sum(x[\"input_len\"]) / sample_size,\n",
        "            sum(x[\"input_longer_512\"]) / sample_size,\n",
        "            sum(x[\"input_longer_256\"]) / sample_size,\n",
        "            sum(x[\"input_longer_128\"]) / sample_size,   \n",
        "            sum(x[\"out_len\"]) / sample_size,\n",
        "            sum(x[\"out_longer_128\"]) / sample_size,\n",
        "            sum(x[\"out_longer_64\"]) / sample_size,\n",
        "            sum(x[\"out_longer_16\"]) / sample_size,\n",
        "        )\n",
        "    )\n",
        "\n",
        "output = data_stats.map(\n",
        "  compute_and_print_stats, \n",
        "  batched=True,\n",
        "  batch_size=-1,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "48n7qnn9NAES"
      },
      "source": [
        "As we can see input and output mean are around 20 tokens. So we should not use less than 20 token in both econder and decoder to keep all the info there. For this example we will use 64, but we could use 32."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ruygyHEAR-BL"
      },
      "source": [
        "### Let's tokenize our data and format the data as expected by the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "ppkB_UwQH7a6"
      },
      "outputs": [],
      "source": [
        "# tokenize the examples\n",
        "def convert_to_features(example_batch):\n",
        "    input_encodings = tokenizer.batch_encode_plus(example_batch['input'], truncation=True, padding='max_length', max_length=512)\n",
        "    target_encodings = tokenizer.batch_encode_plus(example_batch['target'], truncation=True, padding='max_length', max_length=64)\n",
        "\n",
        "    encodings = {\n",
        "        'input_ids': input_encodings['input_ids'], \n",
        "        'attention_mask': input_encodings['attention_mask'],\n",
        "        'labels': target_encodings['input_ids'],\n",
        "        'decoder_attention_mask': target_encodings['attention_mask']\n",
        "    }\n",
        "\n",
        "    return encodings"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 104,
          "referenced_widgets": [
            "8df570109ae44ed58643d60fbf28d992",
            "88b1624e344846ec9ef0e56c15481aa8",
            "cd1c493f3e9e475f93bfcaa2682824e0",
            "ac5da44bdb2d44f6b3732ac2f8ad2f23",
            "cb53dccd1d2b4912808ab5a849d7cbac",
            "c81fbb575fd8489eb62c1af4a859135e",
            "fae5436ebc9b4b5c9cc34c16d6892436",
            "b8d6535079f040d6b966cd824de41de2",
            "0413c1aeb87947769960174b8ccf0b1f",
            "3fe76680764a40b0a2b6467130404f52",
            "3ff0bc02b41449dbaa8ac38158908dd8"
          ]
        },
        "id": "JxPcQH1aIEj_",
        "outputId": "eb33bdee-c6c6-4248-d5ee-af774090bca2"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Loading cached processed dataset at /root/.cache/huggingface/datasets/asahi417___qg_squad/default/0.0.0/6e170fd3cee937658059107ad53d4b6da51416c7c3e2cddbb3c8ab57b4fd1bd5/cache-5da5d56a57445ed3.arrow\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/11 [00:00<?, ?ba/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "8df570109ae44ed58643d60fbf28d992"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Loading cached processed dataset at /root/.cache/huggingface/datasets/asahi417___qg_squad/default/0.0.0/6e170fd3cee937658059107ad53d4b6da51416c7c3e2cddbb3c8ab57b4fd1bd5/cache-256827c787ed8a5c.arrow\n"
          ]
        }
      ],
      "source": [
        "dataset = dataset.map(convert_to_features, batched=True, remove_columns=dataset['train'].column_names)\n",
        "\n",
        "columns = ['input_ids', 'attention_mask', 'labels', 'decoder_attention_mask']\n",
        "\n",
        "dataset.set_format(type='torch', columns=columns)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "wRNxG52qIYMf"
      },
      "outputs": [],
      "source": [
        "from transformers import Seq2SeqTrainer\n",
        "from transformers import Seq2SeqTrainingArguments"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A2cbQnUJSG4i"
      },
      "source": [
        "### Set the training arguments"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "ZH-HHDI0IWrX"
      },
      "outputs": [],
      "source": [
        "# set training arguments - Feel free to adapt it\n",
        "training_args = Seq2SeqTrainingArguments(\n",
        "    output_dir=\"t5-small-finetuned-qgsquad-qgen\",\n",
        "    per_device_train_batch_size=16,\n",
        "    num_train_epochs=5,\n",
        "    per_device_eval_batch_size=16,\n",
        "    predict_with_generate=True,\n",
        "    evaluation_strategy=\"epoch\",\n",
        "    do_train=True,\n",
        "    do_eval=True,\n",
        "    logging_steps=500,\n",
        "    save_strategy=\"epoch\",\n",
        "    #save_steps=1000,\n",
        "    #eval_steps=1000,\n",
        "    overwrite_output_dir=True,\n",
        "    save_total_limit=3,\n",
        "    load_best_model_at_end=True,\n",
        "    push_to_hub=True\n",
        "    #fp16=True, \n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0FrvqN66N20E"
      },
      "source": [
        "### Let's define our metrics to track our training progress. In this case we will use `Rouge`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "SqWbevh5Kelo"
      },
      "outputs": [],
      "source": [
        "rouge = load_metric(\"rouge\")\n",
        "\n",
        "def compute_metrics(pred):\n",
        "    labels_ids = pred.label_ids\n",
        "    pred_ids = pred.predictions\n",
        "\n",
        "    # all unnecessary tokens are removed\n",
        "    pred_str = tokenizer.batch_decode(pred_ids, skip_special_tokens=True)\n",
        "    labels_ids[labels_ids == -100] = tokenizer.pad_token_id\n",
        "    label_str = tokenizer.batch_decode(labels_ids, skip_special_tokens=True)\n",
        "\n",
        "    rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=[\"rouge4\"])[\"rouge4\"].mid\n",
        "\n",
        "    return {\n",
        "        \"rouge4_precision\": round(rouge_output.precision, 4),\n",
        "        \"rouge4_recall\": round(rouge_output.recall, 4),\n",
        "        \"rouge4_fmeasure\": round(rouge_output.fmeasure, 4),\n",
        "    }"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y-Q4lxu8SMGo"
      },
      "source": [
        "### Let's create a `Trainer` instance"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IAbtaix-KnXC",
        "outputId": "0981750e-9b92-4fdf-b497-f3e589c39ab8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/content/t5-small-finetuned-qgsquad-qgen is already a clone of https://huggingface.co/mrm8488/t5-small-finetuned-qgsquad-qgen. Make sure you pull the latest changes with `repo.git_pull()`.\n"
          ]
        }
      ],
      "source": [
        "# instantiate trainer\n",
        "trainer = Seq2SeqTrainer(\n",
        "    model=model,\n",
        "    args=training_args,\n",
        "    compute_metrics=compute_metrics,\n",
        "    train_dataset=dataset['train'],\n",
        "    eval_dataset=dataset['validation'],\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UjNABeOSN-dT"
      },
      "source": [
        "Let's run a sanity check"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 211
        },
        "id": "c-MSUMJqKrqb",
        "outputId": "7099443c-42ae-4b90-9d03-a1c0109b90d4"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "***** Running Evaluation *****\n",
            "  Num examples = 10570\n",
            "  Batch size = 16\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='661' max='661' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [661/661 03:03]\n",
              "    </div>\n",
              "    "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'eval_loss': 15.49451732635498,\n",
              " 'eval_rouge4_fmeasure': 0.0308,\n",
              " 'eval_rouge4_precision': 0.0328,\n",
              " 'eval_rouge4_recall': 0.0346,\n",
              " 'eval_runtime': 185.2648,\n",
              " 'eval_samples_per_second': 57.053,\n",
              " 'eval_steps_per_second': 3.568}"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "trainer.evaluate()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z1pKhH-YRQQV"
      },
      "source": [
        "Track the training in real time using `tensorboard`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 839,
          "resources": {
            "https://localhost:6006/?tensorboardColab=true": {
              "data": "[encoded data, 252024 characters]",
              "headers": [
                [
                  "content-type",
                  "text/html; charset=utf-8"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/environment": {
              "data": "eyJ2ZXJzaW9uIjogIjIuOC4wIiwgImRhdGFfbG9jYXRpb24iOiAiL2NvbnRlbnQvdDUtc21hbGwtZmluZXR1bmVkLXFnc3F1YWQtcWdlbi9ydW5zIiwgIndpbmRvd190aXRsZSI6ICIiLCAiZXhwZXJpbWVudF9uYW1lIjogIiIsICJleHBlcmltZW50X2Rlc2NyaXB0aW9uIjogIiIsICJjcmVhdGlvbl90aW1lIjogMC4wLCAiZGVidWciOiB7ImRhdGFfcHJvdmlkZXIiOiAiR3JwY0RhdGFQcm92aWRlcihhZGRyPSdsb2NhbGhvc3Q6NDUyMzknKSIsICJmbGFncyI6IHsibG9nZGlyIjogIi9jb250ZW50L3Q1LXNtYWxsLWZpbmV0dW5lZC1xZ3NxdWFkLXFnZW4vcnVucyIsICJsb2dkaXJfc3BlYyI6ICIiLCAiaG9zdCI6IG51bGwsICJiaW5kX2FsbCI6IGZhbHNlLCAicG9ydCI6IG51bGwsICJyZXVzZV9wb3J0IjogZmFsc2UsICJsb2FkX2Zhc3QiOiAiYXV0byIsICJleHRyYV9kYXRhX3NlcnZlcl9mbGFncyI6ICIiLCAiZ3JwY19jcmVkc190eXBlIjogImxvY2FsIiwgImdycGNfZGF0YV9wcm92aWRlciI6ICIiLCAicHVyZ2Vfb3JwaGFuZWRfZGF0YSI6IHRydWUsICJkYiI6ICIiLCAiZGJfaW1wb3J0IjogZmFsc2UsICJpbnNwZWN0IjogZmFsc2UsICJ2ZXJzaW9uX3RiIjogZmFsc2UsICJ0YWciOiAiIiwgImV2ZW50X2ZpbGUiOiAiIiwgInBhdGhfcHJlZml4IjogIiIsICJ3aW5kb3dfdGl0bGUiOiAiIiwgIm1heF9yZWxvYWRfdGhyZWFkcyI6IDEsICJyZWxvYWRfaW50ZXJ2YWwiOiA1LjAsICJyZWxvYWRfdGFzayI6ICJhdXRvIiwgInJlbG9hZF9tdWx0aWZpbGUiOiBudWxsLCAicmVsb2FkX211bHRpZmlsZV9pbmFjdGl2ZV9zZWNzIjogODY0MDAsICJnZW5lcmljX2RhdGEiOiAiYXV0byIsICJzYW1wbGVzX3Blcl9wbHVnaW4iOiB7fSwgImN1c3RvbV9wcmVkaWN0X2ZuIjogIiIsICJ3aXRfZGF0YV9kaXIiOiAiIiwgIl9fdGVuc29yYm9hcmRfc3ViY29tbWFuZCI6ICJzZXJ2ZSJ9fX0=",
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/tags": {
              "data": "eyJNYXkyNF8xNS0zMi01OF81MTU2MGM2MWQ1OTEiOiB7ImV2YWwvcm91Z2U0X3JlY2FsbCI6IHsiZGlzcGxheU5hbWUiOiAiIiwgImRlc2NyaXB0aW9uIjogIiJ9LCAiZXZhbC9zYW1wbGVzX3Blcl9zZWNvbmQiOiB7ImRpc3BsYXlOYW1lIjogIiIsICJkZXNjcmlwdGlvbiI6ICIifSwgInRyYWluL2xvc3MiOiB7ImRpc3BsYXlOYW1lIjogIiIsICJkZXNjcmlwdGlvbiI6ICIifSwgInRyYWluL3RyYWluX3NhbXBsZXNfcGVyX3NlY29uZCI6IHsiZGlzcGxheU5hbWUiOiAiIiwgImRlc2NyaXB0aW9uIjogIiJ9LCAidHJhaW4vbGVhcm5pbmdfcmF0ZSI6IHsiZGlzcGxheU5hbWUiOiAiIiwgImRlc2NyaXB0aW9uIjogIiJ9LCAiZXZhbC9ydW50aW1lIjogeyJkaXNwbGF5TmFtZSI6ICIiLCAiZGVzY3JpcHRpb24iOiAiIn0sICJldmFsL3N0ZXBzX3Blcl9zZWNvbmQiOiB7ImRpc3BsYXlOYW1lIjogIiIsICJkZXNjcmlwdGlvbiI6ICIifSwgInRyYWluL3RyYWluX3N0ZXBzX3Blcl9zZWNvbmQiOiB7ImRpc3BsYXlOYW1lIjogIiIsICJkZXNjcmlwdGlvbiI6ICIifSwgImV2YWwvcm91Z2U0X3ByZWNpc2lvbiI6IHsiZGlzcGxheU5hbWUiOiAiIiwgImRlc2NyaXB0aW9uIjogIiJ9LCAiZXZhbC9sb3NzIjogeyJkaXNwbGF5TmFtZSI6ICIiLCAiZGVzY3JpcHRpb24iOiAiIn0sICJ0cmFpbi90cmFpbl9ydW50aW1lIjogeyJkaXNwbGF5TmFtZSI6ICIiLCAiZGVzY3JpcHRpb24iOiAiIn0sICJ0cmFpbi90cmFpbl9sb3NzIjogeyJkaXNwbGF5TmFtZSI6ICIiLCAiZGVzY3JpcHRpb24iOiAiIn0sICJ0cmFpbi90b3RhbF9mbG9zIjogeyJkaXNwbGF5TmFtZSI6ICIiLCAiZGVzY3JpcHRpb24iOiAiIn0sICJldmFsL3JvdWdlNF9mbWVhc3VyZSI6IHsiZGlzcGxheU5hbWUiOiAiIiwgImRlc2NyaXB0aW9uIjogIiJ9LCAidHJhaW4vZXBvY2giOiB7ImRpc3BsYXlOYW1lIjogIiIsICJkZXNjcmlwdGlvbiI6ICIifX0sICJNYXkyNF8xNS0yNy00M181MTU2MGM2MWQ1OTEiOiB7ImV2YWwvcnVudGltZSI6IHsiZGlzcGxheU5hbWUiOiAiIiwgImRlc2NyaXB0aW9uIjogIiJ9LCAiZXZhbC9zYW1wbGVzX3Blcl9zZWNvbmQiOiB7ImRpc3BsYXlOYW1lIjogIiIsICJkZXNjcmlwdGlvbiI6ICIifSwgImV2YWwvc3RlcHNfcGVyX3NlY29uZCI6IHsiZGlzcGxheU5hbWUiOiAiIiwgImRlc2NyaXB0aW9uIjogIiJ9fX0=",
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugins_listing": {
              "data": "eyJzY2FsYXJzIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IHRydWUsICJyZW1vdmVfZG9tIjogZmFsc2UsICJ0YWJfbmFtZSI6ICJzY2FsYXJzIiwgImxvYWRpbmdfbWVjaGFuaXNtIjogeyJ0eXBlIjogIkNVU1RPTV9FTEVNRU5UIiwgImVsZW1lbnRfbmFtZSI6ICJ0Zi1zY2FsYXItZGFzaGJvYXJkIn19LCAiY3VzdG9tX3NjYWxhcnMiOiB7ImRpc2FibGVfcmVsb2FkIjogZmFsc2UsICJlbmFibGVkIjogZmFsc2UsICJyZW1vdmVfZG9tIjogZmFsc2UsICJ0YWJfbmFtZSI6ICJDdXN0b20gU2NhbGFycyIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJDVVNUT01fRUxFTUVOVCIsICJlbGVtZW50X25hbWUiOiAidGYtY3VzdG9tLXNjYWxhci1kYXNoYm9hcmQifX0sICJpbWFnZXMiOiB7ImRpc2FibGVfcmVsb2FkIjogZmFsc2UsICJlbmFibGVkIjogZmFsc2UsICJyZW1vdmVfZG9tIjogZmFsc2UsICJ0YWJfbmFtZSI6ICJpbWFnZXMiLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiQ1VTVE9NX0VMRU1FTlQiLCAiZWxlbWVudF9uYW1lIjogInRmLWltYWdlLWRhc2hib2FyZCJ9fSwgImF1ZGlvIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiYXVkaW8iLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiQ1VTVE9NX0VMRU1FTlQiLCAiZWxlbWVudF9uYW1lIjogInRmLWF1ZGlvLWRhc2hib2FyZCJ9fSwgImRlYnVnZ2VyLXYyIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiRGVidWdnZXIgVjIiLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiTkdfQ09NUE9ORU5UIn19LCAiZ3JhcGhzIjogeyJkaXNhYmxlX3JlbG9hZCI6IHRydWUsICJlbmFibGVkIjogZmFsc2UsICJyZW1vdmVfZG9tIjogZmFsc2UsICJ0YWJfbmFtZSI6ICJncmFwaHMiLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiQ1VTVE9NX0VMRU1FTlQiLCAiZWxlbWVudF9uYW1lIjogInRmLWdyYXBoLWRhc2hib2FyZCJ9fSwgImRpc3RyaWJ1dGlvbnMiOiB7ImRpc2FibGVfcmVsb2FkIjogZmFsc2UsICJlbmFibGVkIjogZmFsc2UsICJyZW1vdmVfZG9tIjogZmFsc2UsICJ0YWJfbmFtZSI6ICJkaXN0cmlidXRpb25zIiwgImxvYWRpbmdfbWVjaGFuaXNtIjogeyJ0eXBlIjogIkNVU1RPTV9FTEVNRU5UIiwgImVsZW1lbnRfbmFtZSI6ICJ0Zi1kaXN0cmlidXRpb24tZGFzaGJvYXJkIn19LCAiaGlzdG9ncmFtcyI6IHsiZGlzYWJsZV9yZWxvYWQiOiBmYWxzZSwgImVuYWJsZWQiOiBmYWxzZSwgInJlbW92ZV9kb20iOiBmYWxzZSwgInRhYl9uYW1lIjogImhpc3RvZ3JhbXMiLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiQ1VTVE9NX0VMRU1FTlQiLCAiZWxlbWVudF9uYW1lIjogInRmLWhpc3RvZ3JhbS1kYXNoYm9hcmQifX0sICJ0ZXh0IjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IHRydWUsICJyZW1vdmVfZG9tIjogZmFsc2UsICJ0YWJfbmFtZSI6ICJ0ZXh0IiwgImxvYWRpbmdfbWVjaGFuaXNtIjogeyJ0eXBlIjogIkNVU1RPTV9FTEVNRU5UIiwgImVsZW1lbnRfbmFtZSI6ICJ0Zi10ZXh0LWRhc2hib2FyZCJ9fSwgInByX2N1cnZlcyI6IHsiZGlzYWJsZV9yZWxvYWQiOiBmYWxzZSwgImVuYWJsZWQiOiBmYWxzZSwgInJlbW92ZV9kb20iOiBmYWxzZSwgInRhYl9uYW1lIjogIlBSIEN1cnZlcyIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJDVVNUT01fRUxFTUVOVCIsICJlbGVtZW50X25hbWUiOiAidGYtcHItY3VydmUtZGFzaGJvYXJkIn19LCAicHJvZmlsZV9yZWRpcmVjdCI6IHsiZGlzYWJsZV9yZWxvYWQiOiBmYWxzZSwgImVuYWJsZWQiOiBmYWxzZSwgInJlbW92ZV9kb20iOiBmYWxzZSwgInRhYl9uYW1lIjogIlByb2ZpbGUiLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiQ1VTVE9NX0VMRU1FTlQiLCAiZWxlbWVudF9uYW1lIjogInRmLXByb2ZpbGUtcmVkaXJlY3QtZGFzaGJvYXJkIn19LCAiaHBhcmFtcyI6IHsiZGlzYWJsZV9yZWxvYWQiOiBmYWxzZSwgImVuYWJsZWQiOiB0cnVlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiaHBhcmFtcyIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJDVVNUT01fRUxFTUVOVCIsICJlbGVtZW50X25hbWUiOiAidGYtaHBhcmFtcy1kYXNoYm9hcmQifX0sICJtZXNoIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAibWVzaCIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJDVVNUT01fRUxFTUVOVCIsICJlbGVtZW50X25hbWUiOiAibWVzaC1kYXNoYm9hcmQifX0sICJ0aW1lc2VyaWVzIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IHRydWUsICJyZW1vdmVfZG9tIjogZmFsc2UsICJ0YWJfbmFtZSI6ICJUaW1lIFNlcmllcyIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJOR19DT01QT05FTlQifX0sICJwcm9qZWN0b3IiOiB7ImRpc2FibGVfcmVsb2FkIjogdHJ1ZSwgImVuYWJsZWQiOiBmYWxzZSwgInJlbW92ZV9kb20iOiBmYWxzZSwgInRhYl9uYW1lIjogInByb2plY3RvciIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJJRlJBTUUiLCAibW9kdWxlX3BhdGgiOiAiL2RhdGEvcGx1Z2luL3Byb2plY3Rvci9pbmRleC5qcyJ9fSwgIndoYXRpZiI6IHsiZGlzYWJsZV9yZWxvYWQiOiBmYWxzZSwgImVuYWJsZWQiOiBmYWxzZSwgInJlbW92ZV9kb20iOiBmYWxzZSwgInRhYl9uYW1lIjogIldoYXQtSWYgVG9vbCIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJJRlJBTUUiLCAibW9kdWxlX3BhdGgiOiAiL2RhdGEvcGx1Z2luL3doYXRpZi9pbmRleC5qcyJ9fX0=",
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/runs": {
              "data": "WyJNYXkyNF8xNS0yNy00M181MTU2MGM2MWQ1OTEiLCAiTWF5MjRfMTUtMzItNThfNTE1NjBjNjFkNTkxIiwgIk1heTI0XzE1LTMyLTU4XzUxNTYwYzYxZDU5MS8xNjUzNDA2NTk4LjMzOTEzODMiXQ==",
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/font-roboto/Hgo13k-tfSpn0qi1SFdUfZBw1xU1rKptJj_0jans920.woff2": {
              "data": "[encoded data, 14504 characters]",
              "headers": [
                [
                  "content-type",
                  "font/woff2"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/font-roboto/RxZJdnzeo3R5zSexge8UUZBw1xU1rKptJj_0jans920.woff2": {
              "data": "[encoded data, 14384 characters]",
              "headers": [
                [
                  "content-type",
                  "font/woff2"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/font-roboto/oMMgfZMQthOryQo9n22dcuvvDin1pK8aKteLpeZ5c0A.woff2": {
              "data": "[encoded data, 14332 characters]",
              "headers": [
                [
                  "content-type",
                  "font/woff2"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/icon_bundle.svg": {
              "data": "[encoded data, 18556 characters]",
              "headers": [
                [
                  "content-type",
                  "image/svg+xml; charset=utf-8"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/index.js?_file_hash=4a366cbe": {
              "data": "[encoded data, 10228016 characters]",
              "headers": [
                [
                  "content-type",
                  "application/javascript; charset=utf-8"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=eval%2Floss": {
              "data": "W1sxNjUzNDA2NTc2LjgwMjg0LCAwLCAxNS40OTQ1MTczMjYzNTQ5OF0sIFsxNjUzNDA3NzUxLjc4NTI0NywgNDczMywgMC4zOTYwMjg3NTcwOTUzMzY5XSwgWzE2NTM0MDg5MjIuMjA5Mjc2MiwgOTQ2NiwgMC4zODYyODc4MDg0MTgyNzM5XSwgWzE2NTM0MTAwNzQuMzc0NDI5LCAxNDE5OSwgMC4zODA1OTkyOTAxMzI1MjI2XSwgWzE2NTM0MTEyMzQuOTk3NTcwMywgMTg5MzIsIDAuMzc4NTgwMjEyNTkzMDc4Nl0sIFsxNjUzNDEyNjAzLjA5ODcwNDMsIDIzNjY1LCAwLjQwMzkxOTMwOTM3NzY3MDNdXQ==",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=eval%2Frouge4_precision": {
              "data": "W1sxNjUzNDA2NTc2LjgwMjkxLCAwLCAwLjAzMjgwMDAwMDE2MDkzMjU0XSwgWzE2NTM0MDc3NTEuNzg1MzUzLCA0NzMzLCAwLjA5ODM5OTk5Njc1NzUwNzMyXSwgWzE2NTM0MDg5MjIuMjA5MzkxNCwgOTQ2NiwgMC4xMDYxMDAwMDA0NDEwNzQzN10sIFsxNjUzNDEwMDc0LjM3NDUxNTUsIDE0MTk5LCAwLjEwNTA5OTk5ODQxNDUxNjQ1XSwgWzE2NTM0MTEyMzQuOTk3NjY0MiwgMTg5MzIsIDAuMTA4NDAwMDAyMTIxOTI1MzVdLCBbMTY1MzQxMjYwMy4wOTg3Njk0LCAyMzY2NSwgMC4wOTMwOTk5OTY0NDc1NjMxN11d",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-27-43_51560c61d591&tag=eval%2Fruntime": {
              "data": "W1sxNjUzNDA2MzA2LjAyNDk0OTYsIDAsIDAuMDEwNDAwMDAwMDI4MzEyMjA2XV0=",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-27-43_51560c61d591&tag=eval%2Fsteps_per_second": {
              "data": "W1sxNjUzNDA2MzA2LjAyNTEyMywgMCwgMC4wXV0=",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=eval%2Fsteps_per_second": {
              "data": "W1sxNjUzNDA2NTc2LjgwMzAwOTcsIDAsIDMuNTY4MDAwMDc4MjAxMjk0XSwgWzE2NTM0MDc3NTEuNzg1NDYsIDQ3MzMsIDMuNjY0OTk5OTYxODUzMDI3M10sIFsxNjUzNDA4OTIyLjIwOTQ5NjUsIDk0NjYsIDMuNjU0MDAwMDQzODY5MDE4Nl0sIFsxNjUzNDEwMDc0LjM3NDYyNSwgMTQxOTksIDMuNjYxOTk5OTQwODcyMTkyNF0sIFsxNjUzNDExMjM0Ljk5Nzc3MTUsIDE4OTMyLCAzLjU5MjAwMDAwNzYyOTM5NDVdLCBbMTY1MzQxMjYwMy4wOTg4NzIsIDIzNjY1LCAzLjYwNDAwMDA5MTU1MjczNDRdXQ==",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-27-43_51560c61d591&tag=eval%2Fsamples_per_second": {
              "data": "W1sxNjUzNDA2MzA2LjAyNTA3MjgsIDAsIDAuMF1d",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=eval%2Frouge4_fmeasure": {
              "data": "W1sxNjUzNDA2NTc2LjgwMjk1NjgsIDAsIDAuMDMwNzk5OTk5ODMzMTA2OTk1XSwgWzE2NTM0MDc3NTEuNzg1NDA0NywgNDczMywgMC4wODg4OTk5OTk4NTY5NDg4NV0sIFsxNjUzNDA4OTIyLjIwOTQ0MTQsIDk0NjYsIDAuMDk2Mjk5OTk4NDYyMjAwMTZdLCBbMTY1MzQxMDA3NC4zNzQ1NywgMTQxOTksIDAuMDk1NDk5OTk5ODIxMTg2MDddLCBbMTY1MzQxMTIzNC45OTc3MTcsIDE4OTMyLCAwLjA5ODYwMDAwMDE0MzA1MTE1XSwgWzE2NTM0MTI2MDMuMDk4ODE3LCAyMzY2NSwgMC4wODQyOTk5OTY0OTUyNDY4OV1d",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=eval%2Fruntime": {
              "data": "W1sxNjUzNDA2NTc2LjgwMjk3NSwgMCwgMTg1LjI2NDgwMTAyNTM5MDYyXSwgWzE2NTM0MDc3NTEuNzg1NDIzOCwgNDczMywgMTgwLjM1NzY5NjUzMzIwMzEyXSwgWzE2NTM0MDg5MjIuMjA5NDYwNSwgOTQ2NiwgMTgwLjg3NTEwNjgxMTUyMzQ0XSwgWzE2NTM0MTAwNzQuMzc0NTg5LCAxNDE5OSwgMTgwLjUxODQwMjA5OTYwOTM4XSwgWzE2NTM0MTEyMzQuOTk3NzM1NywgMTg5MzIsIDE4NC4wMDEyOTY5OTcwNzAzXSwgWzE2NTM0MTI2MDMuMDk4ODM1NywgMjM2NjUsIDIwNi4xNTgyMDMxMjVdXQ==",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=eval%2Frouge4_recall": {
              "data": "W1sxNjUzNDA2NTc2LjgwMjkzNywgMCwgMC4wMzQ2MDAwMDA4Mjg1MDQ1Nl0sIFsxNjUzNDA3NzUxLjc4NTM4NCwgNDczMywgMC4wODY2OTk5OTk4Njg4Njk3OF0sIFsxNjUzNDA4OTIyLjIwOTQyMTYsIDk0NjYsIDAuMDk0NTk5OTk5NDg3NDAwMDVdLCBbMTY1MzQxMDA3NC4zNzQ1NDk5LCAxNDE5OSwgMC4wOTM4MDAwMDA4NDYzODU5Nl0sIFsxNjUzNDExMjM0Ljk5NzY5NjksIDE4OTMyLCAwLjA5NzAwMDAwMjg2MTAyMjk1XSwgWzE2NTM0MTI2MDMuMDk4Nzk3LCAyMzY2NSwgMC4wODM0MDAwMDM2MTIwNDE0N11d",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=eval%2Fsamples_per_second": {
              "data": "W1sxNjUzNDA2NTc2LjgwMjk5MjMsIDAsIDU3LjA1MzAwMTQwMzgwODU5NF0sIFsxNjUzNDA3NzUxLjc4NTQ0MjQsIDQ3MzMsIDU4LjYwNTk5ODk5MjkxOTkyXSwgWzE2NTM0MDg5MjIuMjA5NDc4OSwgOTQ2NiwgNTguNDM3OTk5NzI1MzQxOF0sIFsxNjUzNDEwMDc0LjM3NDYwNywgMTQxOTksIDU4LjU1NDAwMDg1NDQ5MjE5XSwgWzE2NTM0MTEyMzQuOTk3NzUzOSwgMTg5MzIsIDU3LjQ0NDk5OTY5NDgyNDIyXSwgWzE2NTM0MTI2MDMuMDk4ODU0LCAyMzY2NSwgNTcuNjExMDAwMDYxMDM1MTU2XV0=",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/font-roboto/d-6IYplOFocCacKzxwXSOJBw1xU1rKptJj_0jans920.woff2": {
              "data": "[encoded data, 14352 characters]",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "font/woff2"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=train%2Fepoch": {
              "data": "W1sxNjUzNDA2NzAxLjE5MjEwMzEsIDUwMCwgMC4xMDk5OTk5OTk0MDM5NTM1NV0sIFsxNjUzNDA2ODA0LjA2MjM0MDMsIDEwMDAsIDAuMjA5OTk5OTkzNDQzNDg5MDddLCBbMTY1MzQwNjkwNy4wNDA4NDc1LCAxNTAwLCAwLjMxOTk5OTk5Mjg0NzQ0MjZdLCBbMTY1MzQwNzAxMC40NTQxMTE4LCAyMDAwLCAwLjQxOTk5OTk4Njg4Njk3ODE1XSwgWzE2NTM0MDcxMTMuNTAyNjczMSwgMjUwMCwgMC41Mjk5OTk5NzEzODk3NzA1XSwgWzE2NTM0MDcyMTYuMzg0NDk5MywgMzAwMCwgMC42Mjk5OTk5OTUyMzE2Mjg0XSwgWzE2NTM0MDczMTkuMDkxODI3OSwgMzUwMCwgMC43NDAwMDAwMDk1MzY3NDMyXSwgWzE2NTM0MDc0MjEuNDg5MzAzNCwgNDAwMCwgMC44NTAwMDAwMjM4NDE4NTc5XSwgWzE2NTM0MDc1MjMuODEzMzI1MiwgNDUwMCwgMC45NDk5OTk5ODgwNzkwNzFdLCBbMTY1MzQwNzc1MS43ODU0Nzc5LCA0NzMzLCAxLjBdLCBbMTY1MzQwNzgxOC42NTM5MzYsIDUwMDAsIDEuMDU5OTk5OTQyNzc5NTQxXSwgWzE2NTM0MDc5MjYuMjY4MTg4NSwgNTUwMCwgMS4xNTk5OTk5NjY2MjEzOTldLCBbMTY1MzQwODAzMC44ODQ5MDM3LCA2MDAwLCAxLjI2OTk5OTk4MDkyNjUxMzddLCBbMTY1MzQwODEzMy43MzAwMzgsIDY1MDAsIDEuMzcwMDAwMDA0NzY4MzcxNl0sIFsxNjUzNDA4MjM2LjI4NTQ0MywgNzAwMCwgMS40ODAwMDAwMTkwNzM0ODYzXSwgWzE2NTM0MDgzMzguNzI5MTIwNSwgNzUwMCwgMS41ODAwMDAwNDI5MTUzNDQyXSwgWzE2NTM0MDg0NDEuMTY0NDg2NCwgODAwMCwgMS42OTAwMDAwNTcyMjA0NTldLCBbMTY1MzQwODU0My41MzA0NzI1LCA4NTAwLCAxLjc5OTk5OTk1MjMxNjI4NDJdLCBbMTY1MzQwODY0NS45NDgzNzA3LCA5MDAwLCAxLjg5OTk5OTk3NjE1ODE0Ml0sIFsxNjUzNDA4OTIyLjIwOTUxNDQsIDk0NjYsIDIuMF0sIFsxNjUzNDA4OTMxLjY4MzkwMDgsIDk1MDAsIDIuMDA5OTk5OTkwNDYzMjU3XSwgWzE2NTM0MDkwMzQuMDQxMzY4MiwgMTAwMDAsIDIuMTA5OTk5ODk1MDk1ODI1XSwgWzE2NTM0MDkxMzYuNDgyMTY5MiwgMTA1MDAsIDIuMjIwMDAwMDI4NjEwMjI5NV0sIFsxNjUzNDA5MjM4Ljg2OTYyOTQsIDExMDAwLCAyLjMxOTk5OTkzMzI0Mjc5OF0sIFsxNjUzNDA5MzQxLjIyOTkzNTIsIDExNTAwLCAyLjQzMDAwMDA2Njc1NzIwMl0sIFsxNjUzNDA5NDQzLjY3MjE3OSwgMTIwMDAsIDIuNTM5OTk5OTYxODUzMDI3M10sIFsxNjUzNDA5NTQ2LjAzOTE1NTcsIDEyNTAwLCAyLjY0MDAwMDEwNDkwNDE3NV0sIFsxNjUzNDA5NjQ4LjUwMTc2MjQsIDEzMDAwLCAyLjc1XSwgWzE2NTM0MDk3NTAuODM1OTMxMywgMTM1MDAsIDIuODQ5OTk5OTA0NjMyNTY4NF0sIFsxNjUzNDA5ODUzLjE3NzczMSwgMTQwMDAsIDIuOTYwMDAwMDM4MTQ2OTcyN10sIFsxNjUzNDEwMDc0LjM3NDY0MjgsIDE0MTk5LCAzLjBdLCBbMTY1MzQxMDEzOC40NTk1ODExLCAxNDUwMCwgMy4wNTk5OTk5NDI3Nzk1NDFdLCBbMTY1MzQxMDI0MC44NDYxNTQ1LCAxNTAwMCwgMy4xNzAwMDAwNzYyOTM5NDUzXSwgWzE2NTM0MTAzNDMuMzU5OTkxNiwgMTU1MDAsIDMuMjY5OTk5OTgwOTI2NTEzN10sIFsxNjUzNDEwNDQ2Ljg4Mjk2NTMsIDE2MDAwLCAzLjM4MDAwMDExNDQ0MDkxOF0sIFsxNjUzNDEwNTUwLjY2NTgyMzUsIDE2NTAwLCAzLjQ5MDAwMDAwOTUzNjc0M10sIFsxNjUzNDEwNjUzLjgwMzkwMTcsIDE3MDAwLCAzLjU4OTk5OTkxNDE2OTMxMTVdLCBbMTY1MzQxMDc1Ni41MDg3Njc2LCAxNzUwMCwgMy43MDAwMDAwNDc2ODM3MTZdLCBbMTY1MzQxMDg1OS4yNDQ4Mzc1LCAxODAwMCwgMy43OTk5OTk5NTIzMTYyODRdLCBbMTY1MzQxMDk2Mi4yMzY3ODY0LCAxODUwMCwgMy45MTAwMDAwODU4MzA2ODg1XSwgWzE2NTM0MTEyMzQuOTk3Nzg5NiwgMTg5MzIsIDQuMF0sIFsxNjUzNDExMjUxLjMzMjMzMDcsIDE5MDAwLCA0LjAxMDAwMDIyODg4MTgzNl0sIFsxNjUzNDExMzUzLjk2NjU0NSwgMTk1MDAsIDQuMTE5OTk5ODg1NTU5MDgyXSwgWzE2NTM0MTE0NTYuNjk2MTU3NywgMjAwMDAsIDQuMjMwMDAwMDE5MDczNDg2XSwgWzE2NTM0MTE1NTkuNDAyNTM5MywgMjA1MDAsIDQuMzI5OTk5OTIzNzA2MDU1XSwgWzE2NTM0MTE2NjIuMjk0MTg3OCwgMjEwMDAsIDQuNDQwMDAwMDU3MjIwNDU5XSwgWzE2NTM0MTE3NjUuMDU3NzY3NiwgMjE1MDAsIDQuNTM5OTk5OTYxODUzMDI3XSwgWzE2NTM0MTE4NjguMDM5NTI0OCwgMjIwMDAsIDQuNjUwMDAwMDk1MzY3NDMyXSwgWzE2NTM0MTE5NzAuNzE1MDc3NCwgMjI1MDAsIDQuNzVdLCBbMTY1MzQxMjA3My4zODEwNzIzLCAyMzAwMCwgNC44NjAwMDAxMzM1MTQ0MDRdLCBbMTY1MzQxMjE3Ni41MjI5NjEsIDIzNTAwLCA0Ljk2OTk5OTc5MDE5MTY1XSwgWzE2NTM0MTI2MDMuMDk4ODg5OCwgMjM2NjUsIDUuMF1d",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=train%2Floss": {
              "data": "W1sxNjUzNDA2NzAxLjE5MTk5LCA1MDAsIDAuNzM1NzAwMDExMjUzMzU2OV0sIFsxNjUzNDA2ODA0LjA2MjE2LCAxMDAwLCAwLjQ4MjQ5OTk4Njg4Njk3ODE1XSwgWzE2NTM0MDY5MDcuMDQwNjM1LCAxNTAwLCAwLjQ2MDk5OTk5NTQ3MDA0N10sIFsxNjUzNDA3MDEwLjQ1Mzk2MzgsIDIwMDAsIDAuNDUxOTk5OTkyMTMyMTg2OV0sIFsxNjUzNDA3MTEzLjUwMjUxOSwgMjUwMCwgMC40NTA4OTk5ODg0MTI4NTcwNl0sIFsxNjUzNDA3MjE2LjM4NDMzNTUsIDMwMDAsIDAuNDQ1MTAwMDA5NDQxMzc1NzNdLCBbMTY1MzQwNzMxOS4wOTE2NzY1LCAzNTAwLCAwLjQzODE5OTk5Njk0ODI0MjJdLCBbMTY1MzQwNzQyMS40ODkxMzU1LCA0MDAwLCAwLjQzOTUwMDAwNDA1MzExNTg0XSwgWzE2NTM0MDc1MjMuODEzMTU4LCA0NTAwLCAwLjQzMjUwMDAwNDc2ODM3MTZdLCBbMTY1MzQwNzgxOC42NTMwODk4LCA1MDAwLCAwLjQzMDQwMDAxMzkyMzY0NV0sIFsxNjUzNDA3OTI2LjI2Nzk3MSwgNTUwMCwgMC40MTk0OTk5OTMzMjQyNzk4XSwgWzE2NTM0MDgwMzAuODg0NzUxOCwgNjAwMCwgMC40MTY5MDAwMDg5MTY4NTQ4Nl0sIFsxNjUzNDA4MTMzLjcyOTkyNTQsIDY1MDAsIDAuNDE4NzAwMDA5NTg0NDI2OV0sIFsxNjUzNDA4MjM2LjI4NTI2OSwgNzAwMCwgMC40MTc4OTk5OTYwNDIyNTE2XSwgWzE2NTM0MDgzMzguNzI4OTcyMiwgNzUwMCwgMC40MTQ5OTk5OTE2NTUzNDk3M10sIFsxNjUzNDA4NDQxLjE2NDMzMjQsIDgwMDAsIDAuNDE0Nzk5OTg4MjY5ODA1OV0sIFsxNjUzNDA4NTQzLjUzMDMxNDcsIDg1MDAsIDAuNDEyMjAwMDAzODYyMzgxXSwgWzE2NTM0MDg2NDUuOTQ3ODM2OSwgOTAwMCwgMC40MTM3MDAwMTQzNTI3OTg0Nl0sIFsxNjUzNDA4OTMxLjY4MzcxNiwgOTUwMCwgMC40MDYxOTk5OTE3MDMwMzM0NV0sIFsxNjUzNDA5MDM0LjA0MTIyOTUsIDEwMDAwLCAwLjQwMDQwMDAxMjczMTU1MjFdLCBbMTY1MzQwOTEzNi40ODE5MjI2LCAxMDUwMCwgMC40MDY1OTk5OTg0NzQxMjExXSwgWzE2NTM0MDkyMzguODY5NDk3MywgMTEwMDAsIDAuMzk5MTAwMDA1NjI2Njc4NDddLCBbMTY1MzQwOTM0MS4yMjg3MjE0LCAxMTUwMCwgMC40MDE4MDAwMDY2MjgwMzY1XSwgWzE2NTM0MDk0NDMuNjcyMDIxMiwgMTIwMDAsIDAuMzk5ODAwMDAyNTc0OTIwNjVdLCBbMTY1MzQwOTU0Ni4wMzkwMTksIDEyNTAwLCAwLjM5ODc5OTk4NTY0NzIwMTU0XSwgWzE2NTM0MDk2NDguNTAxNjEzOSwgMTMwMDAsIDAuNDAwMDAwMDA1OTYwNDY0NV0sIFsxNjUzNDA5NzUwLjgzNTc1MDMsIDEzNTAwLCAwLjQwNTgwMDAxNDczNDI2ODJdLCBbMTY1MzQwOTg1My4xNzc1NzEsIDE0MDAwLCAwLjM5MTQwMDAwOTM5MzY5Ml0sIFsxNjUzNDEwMTM4LjQ1OTQ0OSwgMTQ1MDAsIDAuMzk3MTk5OTg4MzY1MTczMzRdLCBbMTY1MzQxMDI0MC44NDYwMDQyLCAxNTAwMCwgMC4zOTY0MDAwMDQ2MjUzMjA0M10sIFsxNjUzNDEwMzQzLjM1OTgzOSwgMTU1MDAsIDAuMzk0NDk5OTg3MzYzODE1M10sIFsxNjUzNDEwNDQ2Ljg4MjgwNTgsIDE2MDAwLCAwLjM5MTk5OTk4OTc0ODAwMTFdLCBbMTY1MzQxMDU1MC42NjU2NjczLCAxNjUwMCwgMC4zODgwMDAwMTE0NDQwOTE4XSwgWzE2NTM0MTA2NTMuODAzNzMzLCAxNzAwMCwgMC4zODgzMDAwMDE2MjEyNDYzNF0sIFsxNjUzNDEwNzU2LjUwODYxNzQsIDE3NTAwLCAwLjM5NTAwMDAxMDcyODgzNjA2XSwgWzE2NTM0MTA4NTkuMjQ0NjcwMiwgMTgwMDAsIDAuMzkyNTAwMDEzMTEzMDIxODVdLCBbMTY1MzQxMDk2Mi4yMzY2MTY0LCAxODUwMCwgMC4zOTQ2MDAwMDM5NTc3NDg0XSwgWzE2NTM0MTEyNTEuMzMyMTY3MSwgMTkwMDAsIDAuMzkwNTk5OTk1ODUxNTE2N10sIFsxNjUzNDExMzUzLjk2NjM5NTQsIDE5NTAwLCAwLjM4MjY5OTk5NjIzMjk4NjQ1XSwgWzE2NTM0MTE0NTYuNjk2MDE5LCAyMDAwMCwgMC4zODc2OTk5OTE0NjQ2MTQ4N10sIFsxNjUzNDExNTU5LjQwMjM4MDUsIDIwNTAwLCAwLjM4OTg5OTk5ODkwMzI3NDU0XSwgWzE2NTM0MTE2NjIuMjk0MDE3OCwgMjEwMDAsIDAuMzg2OTAwMDA3NzI0NzYxOTZdLCBbMTY1MzQxMTc2NS4wNTc2MzEsIDIxNTAwLCAwLjM5MjEwMDAwNjM0MTkzNDJdLCBbMTY1MzQxMTg2OC4wMzkzNDY3LCAyMjAwMCwgMC4zODE3MDAwMDkxMDc1ODk3XSwgWzE2NTM0MTE5NzAuNzE0OTIzNiwgMjI1MDAsIDAuMzkxNjk5OTk5NTcwODQ2NTZdLCBbMTY1MzQxMjA3My4zODA5MTQsIDIzMDAwLCAwLjM4NzEwMDAxMTExMDMwNThdLCBbMTY1MzQxMjE3Ni41MjI4MDk1LCAyMzUwMCwgMC4zODU2OTk5ODc0MTE0OTldXQ==",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=train%2Flearning_rate": {
              "data": "W1sxNjUzNDA2NzAxLjE5MjA3NSwgNTAwLCA0Ljg5NDM1ODc4OTg5NzUyNGUtMDVdLCBbMTY1MzQwNjgwNC4wNjIyODExLCAxMDAwLCA0Ljc4ODcxNzM0MjMwNzc5MWUtMDVdLCBbMTY1MzQwNjkwNy4wNDA3OTY4LCAxNTAwLCA0LjY4MzA3NjI1ODUxNTkzOWUtMDVdLCBbMTY1MzQwNzAxMC40NTQwNzA4LCAyMDAwLCA0LjU3NzQzNTE3NDcyNDA4N2UtMDVdLCBbMTY1MzQwNzExMy41MDI2MzE0LCAyNTAwLCA0LjQ3MTc5MzcyNzEzNDM1NDRlLTA1XSwgWzE2NTM0MDcyMTYuMzg0NDU1MiwgMzAwMCwgNC4zNjYxNTI2NDMzNDI1MDI0ZS0wNV0sIFsxNjUzNDA3MzE5LjA5MTc4NTIsIDM1MDAsIDQuMjYwNTExMTk1NzUyNzdlLTA1XSwgWzE2NTM0MDc0MjEuNDg5MjM2OCwgNDAwMCwgNC4xNTQ4NzAxMTE5NjA5MThlLTA1XSwgWzE2NTM0MDc1MjMuODEzMjY2OCwgNDUwMCwgNC4wNDkyMjg2NjQzNzExODVlLTA1XSwgWzE2NTM0MDc4MTguNjUzMjIzOCwgNTAwMCwgMy45NDM1ODc1ODA1NzkzMzNlLTA1XSwgWzE2NTM0MDc5MjYuMjY4MTI4OSwgNTUwMCwgMy44Mzc5NDY0OTY3ODc0ODFlLTA1XSwgWzE2NTM0MDgwMzAuODg0ODY0LCA2MDAwLCAzLjczMjMwNTA0OTE5Nzc0OGUtMDVdLCBbMTY1MzQwODEzMy43MzAwMSwgNjUwMCwgMy42MjY2NjM5NjU0MDU4OTZlLTA1XSwgWzE2NTM0MDgyMzYuMjg1Mzk5MiwgNzAwMCwgMy41MjEwMjI1MTc4MTYxNjM2ZS0wNV0sIFsxNjUzNDA4MzM4LjcyOTA4MTIsIDc1MDAsIDMuNDE1MzgxNDM0MDI0MzExNmUtMDVdLCBbMTY1MzQwODQ0MS4xNjQ0NDU5LCA4MDAwLCAzLjMwOTczOTk4NjQzNDU3OWUtMDVdLCBbMTY1MzQwODU0My41MzA0MjUzLCA4NTAwLCAzLjIwNDA5ODkwMjY0MjcyN2UtMDVdLCBbMTY1MzQwODY0NS45NDgyNjYzLCA5MDAwLCAzLjA5ODQ1NzgxODg1MDg3NWUtMDVdLCBbMTY1MzQwODkzMS42ODM4NTEyLCA5NTAwLCAyLjk5MjgxNjM3MTI2MTE0MjJlLTA1XSwgWzE2NTM0MDkwMzQuMDQxMzM4NywgMTAwMDAsIDIuODg3MTc1MTA1NTcwMzVlLTA1XSwgWzE2NTM0MDkxMzYuNDgyMDkxNCwgMTA1MDAsIDIuNzgxNTMzODM5ODc5NTU3NWUtMDVdLCBbMTY1MzQwOTIzOC44Njk1ODgsIDExMDAwLCAyLjY3NTg5Mjc1NjA4NzcwNTVlLTA1XSwgWzE2NTM0MDkzNDEuMjI4ODMyMiwgMTE1MDAsIDIuNTcwMjUxNDkwMzk2OTEzZS0wNV0sIFsxNjUzNDA5NDQzLjY3MjEzNDksIDEyMDAwLCAyLjQ2NDYxMDIyNDcwNjEyMDhlLTA1XSwgWzE2NTM0MDk1NDYuMDM5MTE2NiwgMTI1MDAsIDIuMzU4OTY4OTU5MDE1MzI4NGUtMDVdLCBbMTY1MzQwOTY0OC41MDE3MjE2LCAxMzAwMCwgMi4yNTMzMjc2OTMzMjQ1MzZlLTA1XSwgWzE2NTM0MDk3NTAuODM1ODcwMywgMTM1MDAsIDIuMTQ3Njg2NDI3NjMzNzQzN2UtMDVdLCBbMTY1MzQwOTg1My4xNzc2ODY1LCAxNDAwMCwgMi4wNDIwNDUxNjE5NDI5NTE0ZS0wNV0sIFsxNjUzNDEwMTM4LjQ1OTU0LCAxNDUwMCwgMS45MzY0MDM4OTYyNTIxNTllLTA1XSwgWzE2NTM0MTAyNDAuODQ2MTEzNywgMTUwMDAsIDEuODMwNzYyODEyNDYwMzA3ZS0wNV0sIFsxNjUzNDEwMzQzLjM1OTk0ODksIDE1NTAwLCAxLjcyNTEyMTU0Njc2OTUxNDdlLTA1XSwgWzE2NTM0MTA0NDYuODgyOTIyLCAxNjAwMCwgMS42MTk0ODAyODEwNzg3MjIzZS0wNV0sIFsxNjUzNDEwNTUwLjY2NTc3OTQsIDE2NTAwLCAxLjUxMzgzOTAxNTM4NzkzZS0wNV0sIFsxNjUzNDEwNjUzLjgwMzg1MzMsIDE3MDAwLCAxLjQwODE5Nzc0OTY5NzEzNzZlLTA1XSwgWzE2NTM0MTA3NTYuNTA4NzI0LCAxNzUwMCwgMS4zMDI1NTY0ODQwMDYzNDUzZS0wNV0sIFsxNjUzNDEwODU5LjI0NDc5MDYsIDE4MDAwLCAxLjE5NjkxNTMwOTI2NTAyMzFlLTA1XSwgWzE2NTM0MTA5NjIuMjM2NzMxLCAxODUwMCwgMS4wOTEyNzQwNDM1NzQyMzA3ZS0wNV0sIFsxNjUzNDExMjUxLjMzMjI3NjYsIDE5MDAwLCA5Ljg1NjMyNzc3ODgzNDM4NGUtMDZdLCBbMTY1MzQxMTM1My45NjY1MDQ4LCAxOTUwMCwgOC43OTk5MTUxMjE5MjY0NmUtMDZdLCBbMTY1MzQxMTQ1Ni42OTYxMTg0LCAyMDAwMCwgNy43NDM1MDMzNzQ1MTMyMzllLTA2XSwgWzE2NTM0MTE1NTkuNDAyNDkwOSwgMjA1MDAsIDYuNjg3MDkwNzE3NjA1MzE1ZS0wNl0sIFsxNjUzNDExNjYyLjI5NDE0NTYsIDIxMDAwLCA1LjYzMDY3ODA2MDY5NzM5MmUtMDZdLCBbMTY1MzQxMTc2NS4wNTc3MjgzLCAyMTUwMCwgNC41NzQyNjU4NTg1MzY4MTllLTA2XSwgWzE2NTM0MTE4NjguMDM5NDc1LCAyMjAwMCwgMy41MTc4NTM0MjkwMDI1NzFlLTA2XSwgWzE2NTM0MTE5NzAuNzE1MDM3LCAyMjUwMCwgMi40NjE0NDA5OTk0NjgzMjNlLTA2XSwgWzE2NTM0MTIwNzMuMzgxMDI4NCwgMjMwMDAsIDEuNDA1MDI4NTY5OTM0MDc0OGUtMDZdLCBbMTY1MzQxMjE3Ni41MjI5MjAxLCAyMzUwMCwgMy40ODYxNjExMTk3ODExNzI2ZS0wN11d",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=train%2Ftrain_runtime": {
              "data": "W1sxNjUzNDEyMzk2LjkwOTI3MDMsIDIzNjY1LCA1Nzk4LjU4NzQwMjM0Mzc1XV0=",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=train%2Ftotal_flos": {
              "data": "W1sxNjUzNDEyMzk2LjkwOTQ1MiwgMjM2NjUsIDUuMTI0MTc2MDE1OTk1Njk5ZSsxNl1d",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=train%2Ftrain_samples_per_second": {
              "data": "W1sxNjUzNDEyMzk2LjkwOTM5ODYsIDIzNjY1LCA2NS4yOTI5OTkyNjc1NzgxMl1d",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=train%2Ftrain_loss": {
              "data": "W1sxNjUzNDEyMzk2LjkwOTQ3MiwgMjM2NjUsIDAuNDE1MTcwNDAxMzM0NzYyNl1d",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/scalars?run=May24_15-32-58_51560c61d591&tag=train%2Ftrain_steps_per_second": {
              "data": "W1sxNjUzNDEyMzk2LjkwOTQzMDMsIDIzNjY1LCA0LjA4MDk5OTg1MTIyNjgwN11d",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "status": 200,
              "status_text": ""
            }
          }
        },
        "id": "2uiF7tA8KvxQ",
        "outputId": "1854c093-16d9-4264-96ac-c18ecc4b08f7"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Reusing TensorBoard on port 6006 (pid 331), started 0:05:53 ago. (Use '!kill 331' to kill it.)"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ],
            "application/javascript": [
              "\n",
              "        (async () => {\n",
              "            const url = new URL(await google.colab.kernel.proxyPort(6006, {'cache': true}));\n",
              "            url.searchParams.set('tensorboardColab', 'true');\n",
              "            const iframe = document.createElement('iframe');\n",
              "            iframe.src = url;\n",
              "            iframe.setAttribute('width', '100%');\n",
              "            iframe.setAttribute('height', '800');\n",
              "            iframe.setAttribute('frameborder', 0);\n",
              "            document.body.appendChild(iframe);\n",
              "        })();\n",
              "    "
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "%load_ext tensorboard\n",
        "%tensorboard --logdir /content/t5-small-finetuned-qgsquad-qgen/runs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "qZR_ASsEK5Lp",
        "outputId": "18e7c2bc-2af1-4246-9b0c-724bf4530699"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/transformers/optimization.py:309: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
            "  FutureWarning,\n",
            "***** Running training *****\n",
            "  Num examples = 75722\n",
            "  Num Epochs = 5\n",
            "  Instantaneous batch size per device = 16\n",
            "  Total train batch size (w. parallel, distributed & accumulation) = 16\n",
            "  Gradient Accumulation steps = 1\n",
            "  Total optimization steps = 23665\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='23665' max='23665' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [23665/23665 1:36:38, Epoch 5/5]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Epoch</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "      <th>Rouge4 Precision</th>\n",
              "      <th>Rouge4 Recall</th>\n",
              "      <th>Rouge4 Fmeasure</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>0.432500</td>\n",
              "      <td>0.396029</td>\n",
              "      <td>0.098400</td>\n",
              "      <td>0.086700</td>\n",
              "      <td>0.088900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>0.413700</td>\n",
              "      <td>0.386288</td>\n",
              "      <td>0.106100</td>\n",
              "      <td>0.094600</td>\n",
              "      <td>0.096300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>0.391400</td>\n",
              "      <td>0.380599</td>\n",
              "      <td>0.105100</td>\n",
              "      <td>0.093800</td>\n",
              "      <td>0.095500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>0.394600</td>\n",
              "      <td>0.378580</td>\n",
              "      <td>0.108400</td>\n",
              "      <td>0.097000</td>\n",
              "      <td>0.098600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>5</td>\n",
              "      <td>0.385700</td>\n",
              "      <td>0.378448</td>\n",
              "      <td>0.110100</td>\n",
              "      <td>0.099100</td>\n",
              "      <td>0.100700</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "***** Running Evaluation *****\n",
            "  Num examples = 10570\n",
            "  Batch size = 16\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='1322' max='661' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [661/661 22:38]\n",
              "    </div>\n",
              "    "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Saving model checkpoint to t5-small-finetuned-qgsquad-qgen/checkpoint-4733\n",
            "Configuration saved in t5-small-finetuned-qgsquad-qgen/checkpoint-4733/config.json\n",
            "Model weights saved in t5-small-finetuned-qgsquad-qgen/checkpoint-4733/pytorch_model.bin\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 10570\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to t5-small-finetuned-qgsquad-qgen/checkpoint-9466\n",
            "Configuration saved in t5-small-finetuned-qgsquad-qgen/checkpoint-9466/config.json\n",
            "Model weights saved in t5-small-finetuned-qgsquad-qgen/checkpoint-9466/pytorch_model.bin\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 10570\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to t5-small-finetuned-qgsquad-qgen/checkpoint-14199\n",
            "Configuration saved in t5-small-finetuned-qgsquad-qgen/checkpoint-14199/config.json\n",
            "Model weights saved in t5-small-finetuned-qgsquad-qgen/checkpoint-14199/pytorch_model.bin\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 10570\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to t5-small-finetuned-qgsquad-qgen/checkpoint-18932\n",
            "Configuration saved in t5-small-finetuned-qgsquad-qgen/checkpoint-18932/config.json\n",
            "Model weights saved in t5-small-finetuned-qgsquad-qgen/checkpoint-18932/pytorch_model.bin\n",
            "Deleting older checkpoint [t5-small-finetuned-qgsquad-qgen/checkpoint-4733] due to args.save_total_limit\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 10570\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to t5-small-finetuned-qgsquad-qgen/checkpoint-23665\n",
            "Configuration saved in t5-small-finetuned-qgsquad-qgen/checkpoint-23665/config.json\n",
            "Model weights saved in t5-small-finetuned-qgsquad-qgen/checkpoint-23665/pytorch_model.bin\n",
            "Deleting older checkpoint [t5-small-finetuned-qgsquad-qgen/checkpoint-9466] due to args.save_total_limit\n",
            "\n",
            "\n",
            "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
            "\n",
            "\n",
            "Loading best model from t5-small-finetuned-qgsquad-qgen/checkpoint-23665 (score: 0.37844812870025635).\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TrainOutput(global_step=23665, training_loss=0.4151703922183877, metrics={'train_runtime': 5798.5876, 'train_samples_per_second': 65.293, 'train_steps_per_second': 4.081, 'total_flos': 5.124175945531392e+16, 'train_loss': 0.4151703922183877, 'epoch': 5.0})"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 228
        },
        "id": "cwwdZwFX_JdJ",
        "outputId": "0a763c94-c5a2-45ea-d538-3f9af791faf0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "***** Running Evaluation *****\n",
            "  Num examples = 11877\n",
            "  Batch size = 16\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='743' max='743' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [743/743 03:23]\n",
              "    </div>\n",
              "    "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'epoch': 5.0,\n",
              " 'eval_loss': 0.4039193093776703,\n",
              " 'eval_rouge4_fmeasure': 0.0843,\n",
              " 'eval_rouge4_precision': 0.0931,\n",
              " 'eval_rouge4_recall': 0.0834,\n",
              " 'eval_runtime': 206.1582,\n",
              " 'eval_samples_per_second': 57.611,\n",
              " 'eval_steps_per_second': 3.604}"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ],
      "source": [
        "trainer.evaluate(eval_dataset=dataset['test'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 460,
          "referenced_widgets": [
            "4826a263fadb42bb96a655b9db53133a",
            "2d1bf194f11f4011afa013b71930ef4b",
            "5a62ea31f0ca405a9773dec50f65db47",
            "aca809a58df14edf826dee6a9cb1f770",
            "deae9b0000bd4b739bfa111d7834565f",
            "4ea66b71896e41eab8b36938a7987418",
            "f4c6473c781d487cb45e835089332fb5",
            "31ce5ec915554de094c607d469b18e69",
            "751e197802c448dfad695450ff65f3a7",
            "28d06f262040484aa3499de9a309ba98",
            "04ba59fd0ff542cfb0d805e8ed90ee30",
            "69f4b04998564473b9d71d6f4bfe5c1a",
            "97e3906c0cba40bea907c6f6ce883be2",
            "e38105c3925f4361bc23eb5f5844c8f9",
            "5fe771b0963c419da78d4ed1aa2e73e4",
            "79937bf895cc4968aa9d558777f0f44c",
            "a4ec4a140a0a4c17a251616521b3ff64",
            "916f88b5ed9d4986a649be11c7e8c125",
            "3eab5328f2664eb9b75639a4913a5df0",
            "e3e7cafe1da64bcf98b45eda574f511e",
            "f81f143311c346eeb5b82f2aae381a88",
            "23a71ee7af8c4f3191a9246f0935e99f",
            "d35997d8d2f948b5a0ef51c7ef2445bc",
            "c6aeb8f748e54ddda72e2b9695cc3917",
            "aae50c2c22414f7ba20fc5b012be711c",
            "805b522b877c4e798cb4338fb81be4de",
            "00b87471cc52424791d85f1f6a6f1559",
            "f9e7711606a044ebb8436c9592c9dde9",
            "b1561b1d8de6445cbbeb9570941fde7a",
            "b7b7bee99d60493f8f5e2e86285586c4",
            "80b30b9d7a5b40fc9260ce56936a718f",
            "ae9371e4642f4167a0895d6929cd303a",
            "1eb8e84a09d642a5a891385a7c451e83"
          ]
        },
        "id": "M4f_eBTCeTHA",
        "outputId": "913700de-bfe3-45cf-e574-0de1c6dbf578"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Saving model checkpoint to t5-small-finetuned-qgsquad-qgen\n",
            "Configuration saved in t5-small-finetuned-qgsquad-qgen/config.json\n",
            "Model weights saved in t5-small-finetuned-qgsquad-qgen/pytorch_model.bin\n",
            "Saving model checkpoint to t5-small-finetuned-qgsquad-qgen\n",
            "Configuration saved in t5-small-finetuned-qgsquad-qgen/config.json\n",
            "Model weights saved in t5-small-finetuned-qgsquad-qgen/pytorch_model.bin\n",
            "Several commits (2) will be pushed upstream.\n",
            "The progress bars may be unreliable.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Upload file pytorch_model.bin:   0%|          | 3.34k/231M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "4826a263fadb42bb96a655b9db53133a"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Upload file runs/May24_15-32-58_51560c61d591/events.out.tfevents.1653406576.51560c61d591.388.0:  23%|##3      …"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "69f4b04998564473b9d71d6f4bfe5c1a"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Upload file runs/May24_15-32-58_51560c61d591/events.out.tfevents.1653412603.51560c61d591.388.2: 100%|#########…"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d35997d8d2f948b5a0ef51c7ef2445bc"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "remote: Enforcing permissions...        \n",
            "remote: Allowed refs: all        \n",
            "To https://huggingface.co/mrm8488/t5-small-finetuned-qgsquad-qgen\n",
            "   0e3c4b6..50a5981  main -> main\n",
            "\n",
            "Dropping the following result as it does not have all the necessary fields:\n",
            "{'task': {'name': 'Sequence-to-sequence Language Modeling', 'type': 'text2text-generation'}, 'dataset': {'name': 'qg_squad', 'type': 'qg_squad', 'args': 'default'}}\n",
            "remote: Enforcing permissions...        \n",
            "remote: Allowed refs: all        \n",
            "To https://huggingface.co/mrm8488/t5-small-finetuned-qgsquad-qgen\n",
            "   50a5981..756ac4d  main -> main\n",
            "\n"
          ]
        }
      ],
      "source": [
        "trainer.save_model()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5d-QcwKnebYY",
        "outputId": "35f78a64-0192-4443-ca00-7a2cb77e5bc5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "tokenizer config file saved in /content/t5-small-finetuned-qgsquad-qgen/tokenizer_config.json\n",
            "Special tokens file saved in /content/t5-small-finetuned-qgsquad-qgen/special_tokens_map.json\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "('/content/t5-small-finetuned-qgsquad-qgen/tokenizer_config.json',\n",
              " '/content/t5-small-finetuned-qgsquad-qgen/special_tokens_map.json',\n",
              " '/content/t5-small-finetuned-qgsquad-qgen/tokenizer.json')"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ],
      "source": [
        "tokenizer.save_pretrained('/content/t5-small-finetuned-qgsquad-qgen')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JrBFmJbdW9XV",
        "outputId": "49df1a9b-b956-4853-c4af-52e377cf8921"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Dropping the following result as it does not have all the necessary fields:\n",
            "{'task': {'name': 'Sequence-to-sequence Language Modeling', 'type': 'text2text-generation'}, 'dataset': {'name': 'qg_squad', 'type': 'qg_squad', 'args': 'default'}}\n"
          ]
        }
      ],
      "source": [
        "trainer.create_model_card()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nTn9FjGKo0S7",
        "outputId": "86e19f70-88a1-490f-cd9f-6c4169a8827f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Saving model checkpoint to t5-small-finetuned-qgsquad-qgen\n",
            "Configuration saved in t5-small-finetuned-qgsquad-qgen/config.json\n",
            "Model weights saved in t5-small-finetuned-qgsquad-qgen/pytorch_model.bin\n",
            "Dropping the following result as it does not have all the necessary fields:\n",
            "{'task': {'name': 'Sequence-to-sequence Language Modeling', 'type': 'text2text-generation'}, 'dataset': {'name': 'qg_squad', 'type': 'qg_squad', 'args': 'default'}}\n"
          ]
        }
      ],
      "source": [
        "trainer.push_to_hub()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wsJum5nEcHWg"
      },
      "source": [
        "### Test the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "HeaD6sejb7hg",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a952703f-b022-40db-ac61-998b9943aa30"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Didn't find file t5-small-finetuned-qgsquad-qgen/spiece.model. We won't load it.\n",
            "Didn't find file t5-small-finetuned-qgsquad-qgen/added_tokens.json. We won't load it.\n",
            "loading file None\n",
            "loading file t5-small-finetuned-qgsquad-qgen/tokenizer.json\n",
            "loading file None\n",
            "loading file t5-small-finetuned-qgsquad-qgen/special_tokens_map.json\n",
            "loading file t5-small-finetuned-qgsquad-qgen/tokenizer_config.json\n",
            "loading configuration file t5-small-finetuned-qgsquad-qgen/config.json\n",
            "Model config T5Config {\n",
            "  \"_name_or_path\": \"t5-small\",\n",
            "  \"architectures\": [\n",
            "    \"T5ForConditionalGeneration\"\n",
            "  ],\n",
            "  \"d_ff\": 2048,\n",
            "  \"d_kv\": 64,\n",
            "  \"d_model\": 512,\n",
            "  \"decoder_start_token_id\": 0,\n",
            "  \"dropout_rate\": 0.1,\n",
            "  \"eos_token_id\": 1,\n",
            "  \"feed_forward_proj\": \"relu\",\n",
            "  \"initializer_factor\": 1.0,\n",
            "  \"is_encoder_decoder\": true,\n",
            "  \"layer_norm_epsilon\": 1e-06,\n",
            "  \"model_type\": \"t5\",\n",
            "  \"n_positions\": 512,\n",
            "  \"num_decoder_layers\": 6,\n",
            "  \"num_heads\": 8,\n",
            "  \"num_layers\": 6,\n",
            "  \"output_past\": true,\n",
            "  \"pad_token_id\": 0,\n",
            "  \"relative_attention_max_distance\": 128,\n",
            "  \"relative_attention_num_buckets\": 32,\n",
            "  \"task_specific_params\": {\n",
            "    \"summarization\": {\n",
            "      \"early_stopping\": true,\n",
            "      \"length_penalty\": 2.0,\n",
            "      \"max_length\": 200,\n",
            "      \"min_length\": 30,\n",
            "      \"no_repeat_ngram_size\": 3,\n",
            "      \"num_beams\": 4,\n",
            "      \"prefix\": \"summarize: \"\n",
            "    },\n",
            "    \"translation_en_to_de\": {\n",
            "      \"early_stopping\": true,\n",
            "      \"max_length\": 300,\n",
            "      \"num_beams\": 4,\n",
            "      \"prefix\": \"translate English to German: \"\n",
            "    },\n",
            "    \"translation_en_to_fr\": {\n",
            "      \"early_stopping\": true,\n",
            "      \"max_length\": 300,\n",
            "      \"num_beams\": 4,\n",
            "      \"prefix\": \"translate English to French: \"\n",
            "    },\n",
            "    \"translation_en_to_ro\": {\n",
            "      \"early_stopping\": true,\n",
            "      \"max_length\": 300,\n",
            "      \"num_beams\": 4,\n",
            "      \"prefix\": \"translate English to Romanian: \"\n",
            "    }\n",
            "  },\n",
            "  \"torch_dtype\": \"float32\",\n",
            "  \"transformers_version\": \"4.19.2\",\n",
            "  \"use_cache\": true,\n",
            "  \"vocab_size\": 32128\n",
            "}\n",
            "\n",
            "loading weights file t5-small-finetuned-qgsquad-qgen/pytorch_model.bin\n",
            "All model checkpoint weights were used when initializing T5ForConditionalGeneration.\n",
            "\n",
            "All the weights of T5ForConditionalGeneration were initialized from the model checkpoint at t5-small-finetuned-qgsquad-qgen.\n",
            "If your task is similar to the task the model of the checkpoint was trained on, you can already use T5ForConditionalGeneration for predictions without further training.\n"
          ]
        }
      ],
      "source": [
        "CKPT = 't5-small-finetuned-qgsquad-qgen'\n",
        "from transformers import AutoTokenizer, T5ForConditionalGeneration\n",
        "tokenizer = AutoTokenizer.from_pretrained(CKPT)\n",
        "model = T5ForConditionalGeneration.from_pretrained(CKPT)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fGCO9LvFdAG6",
        "outputId": "49f8afe3-caf5-4a25-bd18-0c024fa00abf"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Reusing dataset squad (/root/.cache/huggingface/datasets/squad/plain_text/1.0.0/d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150cce08a837ef62ffea453)\n"
          ]
        }
      ],
      "source": [
        "test_data = load_dataset('squad', split='validation')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "mAMMJJJvcQ38"
      },
      "outputs": [],
      "source": [
        "def generate_question(text):\n",
        "    inputs = tokenizer(text, padding='longest', max_length=512, return_tensors='pt')\n",
        "    input_ids = inputs.input_ids\n",
        "    attention_mask = inputs.attention_mask\n",
        "    output = model.generate(input_ids, attention_mask=attention_mask, max_length=64)\n",
        "\n",
        "    return tokenizer.decode(output[0], skip_special_tokens=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tbuoLD-hcpPF",
        "outputId": "3f1ce311-1041-4a1e-a89f-10af2f139e72"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Real: Which NFL team represented the AFC at Super Bowl 50?\n",
            "Pred: Who defeated Carolina Panthers 24-10 to earn their third Super Bowl title?\n",
            "Answ: Denver Broncos\n",
            "=================================\n",
            "\n",
            "Real: Who were special guests for the Super Bowl halftime show?\n",
            "Pred: Who were the guest performers of the Super Bowl 50 halftime show?\n",
            "Answ: Beyoncé and Bruno Mars\n",
            "=================================\n",
            "\n",
            "Real: Who had the best record in the NFC?\n",
            "Pred: Who was the seventh team to win at least 15 regular season games?\n",
            "Answ: Carolina Panthers\n",
            "=================================\n",
            "\n",
            "Real: Who forced four fumbles for the Broncos in the 2015 season?\n",
            "Pred: Who led the Broncos with 11 sacks?\n",
            "Answ: Von Miller\n",
            "=================================\n",
            "\n",
            "Real: What field type is usually chosen for Super Bowl games?\n",
            "Pred: Where are Super Bowl games played?\n",
            "Answ: natural grass\n",
            "=================================\n",
            "\n",
            "Real: What was media day called for Super Bowl 50?\n",
            "Pred: What was the game's media day re-branded as?\n",
            "Answ: Super Bowl Opening Night.\n",
            "=================================\n",
            "\n",
            "Real: Besides Death Wish Coffee, how many other competitors participated in the contest?\n",
            "Pred: How many other competitors did Death Wish Coffee beat?\n",
            "Answ: nine\n",
            "=================================\n",
            "\n",
            "Real: What did Carolina face in the opening drive that they had not faced the entire postseason?\n",
            "Pred: What was the first time in the entire postseason that Carolina was facing?\n",
            "Answ: a deficit.\n",
            "=================================\n",
            "\n",
            "Real: Who is the only quarterback to win a Super Bowl with two teams?\n",
            "Pred: Who became the oldest quarterback to win a Super Bowl at age 39?\n",
            "Answ: Manning\n",
            "=================================\n",
            "\n",
            "Real: When was Warsaw ranked as the 32nd most liveable city in the world?\n",
            "Pred: When did the Economist Intelligence Unit rank Warsaw as the 32nd most liveable city in the world?\n",
            "Answ: 2012\n",
            "=================================\n",
            "\n"
          ]
        }
      ],
      "source": [
        "for i in range(0,1000,100):\n",
        "  print('Real: ' + test_data[i]['question'])\n",
        "  print('Pred: ' + generate_question('answer: ' + test_data[i]['answers']['text'][0] + ' context: ' + test_data[i]['context']))\n",
        "  print('Answ: ' +  test_data[i]['answers']['text'][0])\n",
        "  print('=================================\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7iZtACF4g1Hq"
      },
      "source": [
        "Custom examples"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "PO1OvqSedZhI",
        "outputId": "bf1d011d-b5f2-41ce-ad30-8af4bc27e3ab"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'What are some scenarios that can be used to generate a valid and fluent question?'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 37
        }
      ],
      "source": [
        "context ='''\n",
        "The goal of Question Generation is to generate a valid and fluent question according to a given passage and the target answer. Question Generation can be used in many scenarios, such as automatic tutoring systems, improving the performance of Question Answering models and enabling chatbots to lead a conversation.\n",
        "'''\n",
        "answer = 'automatic tutoring systems, improving the performance of Question Answering models and enabling chatbots to lead a conversation.'\n",
        "\n",
        "generate_question('answer: ' + answer + ' context: ' + context )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "-V7bmzeQgqtA",
        "outputId": "7e4c7046-3033-48d8-9668-078e21f3b6fd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'What is the aim of Hugging Face?'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 38
        }
      ],
      "source": [
        "context ='''\n",
        "Hugging Face, a company that first built a chat app for bored teens provides open-source NLP technologies, and last year, \n",
        "it raised $15 million to build a definitive NLP library. From its chat app to this day, \n",
        "Hugging Face has been able to swiftly develop language processing expertise. \n",
        "The company’s aim is to advance NLP and democratize it for use by everyone.\n",
        "'''\n",
        "answer = 'advance NLP and democratize it for use by everyone..'\n",
        "\n",
        "generate_question('answer: ' + answer + ' context: ' + context )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MOx78lCspgyI"
      },
      "source": [
        "## Acknowledgments\n",
        "\n",
        "- [Narrativa](https://narrativa.com)\n",
        "- [HuggingFace](https://huggingface.co/mrm8488)\n",
        "- [QBLOCKS](https://www.qblocks.cloud/)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "61dslGdDqHfA"
      },
      "outputs": [],
      "source": [
        ""
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "T5-qgsquad-with-HF-transformers-for_QG.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyN3rVOTYXdkVoNnFex7L/3E",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "634eb7c5672d4e1fa626d5bb1ac2c82b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_30e4b4385f164a1b9a750b4203b1ffb1",
              "IPY_MODEL_d30b3558c4244fa88ee1a560bd0c5a7b",
              "IPY_MODEL_64e7ac4067654e79bd4a4dd39a93f13c"
            ],
            "layout": "IPY_MODEL_bf9a816ff31a4a3883b4c8550ce1820c"
          }
        },
        "30e4b4385f164a1b9a750b4203b1ffb1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5830e80a06cb43dd85a9bb3d941e3737",
            "placeholder": "​",
            "style": "IPY_MODEL_7d8c4f73a8784d04a7b641e0639a4e1b",
            "value": "100%"
          }
        },
        "d30b3558c4244fa88ee1a560bd0c5a7b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_26b3ef6b48f64746b806d112b54a06c3",
            "max": 3,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_20f6a8a53daa4c109a3a6064bb4f6466",
            "value": 3
          }
        },
        "64e7ac4067654e79bd4a4dd39a93f13c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8eb4fb77df6a4388ae4a7d239de9cf0c",
            "placeholder": "​",
            "style": "IPY_MODEL_631cb403f11348dfafd333b9acc9237b",
            "value": " 3/3 [00:00&lt;00:00, 65.16it/s]"
          }
        },
        "bf9a816ff31a4a3883b4c8550ce1820c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5830e80a06cb43dd85a9bb3d941e3737": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7d8c4f73a8784d04a7b641e0639a4e1b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "26b3ef6b48f64746b806d112b54a06c3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "20f6a8a53daa4c109a3a6064bb4f6466": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8eb4fb77df6a4388ae4a7d239de9cf0c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "631cb403f11348dfafd333b9acc9237b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8059c03298d1444ea524d7288d9fafa2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c55b9db0b4c84e0b8bc6592f620435f2",
              "IPY_MODEL_cd488407d9564790aae16ac2664ad89e",
              "IPY_MODEL_47029952f00a4d7a9b53166913612273"
            ],
            "layout": "IPY_MODEL_2a1dd1f17eb048828e159d36a80a84fb"
          }
        },
        "c55b9db0b4c84e0b8bc6592f620435f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_30e1d632b0ad486888286b2f4474e222",
            "placeholder": "​",
            "style": "IPY_MODEL_1214d750cd524ce086258ca9689a91e1",
            "value": "#0: 100%"
          }
        },
        "cd488407d9564790aae16ac2664ad89e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_64b08505aad3482db93c06b93d41a8f0",
            "max": 2500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_29bf66f806c04c58b5d2c20896eaad50",
            "value": 2500
          }
        },
        "47029952f00a4d7a9b53166913612273": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d7b27900fa89460980bee7bd66ba7c69",
            "placeholder": "​",
            "style": "IPY_MODEL_1d6476c677264238843fde427d464496",
            "value": " 2500/2500 [00:09&lt;00:00, 402.66ex/s]"
          }
        },
        "2a1dd1f17eb048828e159d36a80a84fb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "30e1d632b0ad486888286b2f4474e222": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1214d750cd524ce086258ca9689a91e1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "64b08505aad3482db93c06b93d41a8f0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "29bf66f806c04c58b5d2c20896eaad50": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d7b27900fa89460980bee7bd66ba7c69": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1d6476c677264238843fde427d464496": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cad4716558644f788f6afd5b83b37a3d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_28e910bd55cb41459f8a5b5779cacfff",
              "IPY_MODEL_ae8af3448e944352862cd22e723df86d",
              "IPY_MODEL_74cc87c09ee04d5096688d8b54b547da"
            ],
            "layout": "IPY_MODEL_abf7378bff344d6e91ea7fa2d193e735"
          }
        },
        "28e910bd55cb41459f8a5b5779cacfff": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2b45838756d94c0c910ecc8c95badcf4",
            "placeholder": "​",
            "style": "IPY_MODEL_145ee2480bac4ad0b5e015c9e6987433",
            "value": "#3: 100%"
          }
        },
        "ae8af3448e944352862cd22e723df86d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_888d6d377396443dafeb941d6badb025",
            "max": 2500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0bba5c87272742f59767e58e14711351",
            "value": 2500
          }
        },
        "74cc87c09ee04d5096688d8b54b547da": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1be13c9fe339499aa7a837153b613cb5",
            "placeholder": "​",
            "style": "IPY_MODEL_30536dc4f57749db91e890ebbb4c1781",
            "value": " 2500/2500 [00:09&lt;00:00, 423.48ex/s]"
          }
        },
        "abf7378bff344d6e91ea7fa2d193e735": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2b45838756d94c0c910ecc8c95badcf4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "145ee2480bac4ad0b5e015c9e6987433": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "888d6d377396443dafeb941d6badb025": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0bba5c87272742f59767e58e14711351": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1be13c9fe339499aa7a837153b613cb5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "30536dc4f57749db91e890ebbb4c1781": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6092d2342791482da207f191f80a31e9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_3c6a218e5a2f4d0fbe371d5685b7f144",
              "IPY_MODEL_68eece9473c449d09e8df3de5bd83591",
              "IPY_MODEL_b5b079ca962e4e1689426c4d05992f9b"
            ],
            "layout": "IPY_MODEL_3162fb09a87c49d6bc415d346d3f27ae"
          }
        },
        "3c6a218e5a2f4d0fbe371d5685b7f144": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_67aa5863e89b436e9bbffd46396a92b9",
            "placeholder": "​",
            "style": "IPY_MODEL_b1fb7b59a746453db258ea76fe9bbf0f",
            "value": "#1: 100%"
          }
        },
        "68eece9473c449d09e8df3de5bd83591": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c28f26eafae541b6924d3cce9d910ad2",
            "max": 2500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_6c5cbb433b4646e88259b76cffc65925",
            "value": 2500
          }
        },
        "b5b079ca962e4e1689426c4d05992f9b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_df709b3471c64d649e13ef03af161f8d",
            "placeholder": "​",
            "style": "IPY_MODEL_25dfe1f2501d4bcb9d67b7b09d26a73a",
            "value": " 2500/2500 [00:09&lt;00:00, 370.18ex/s]"
          }
        },
        "3162fb09a87c49d6bc415d346d3f27ae": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "67aa5863e89b436e9bbffd46396a92b9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b1fb7b59a746453db258ea76fe9bbf0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c28f26eafae541b6924d3cce9d910ad2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6c5cbb433b4646e88259b76cffc65925": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "df709b3471c64d649e13ef03af161f8d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "25dfe1f2501d4bcb9d67b7b09d26a73a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5d89e1e341714c5fabd3b5452b82bcb6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a0649663e0064c3898698e6ac70f8f0b",
              "IPY_MODEL_e7fcf56d72ae4db2b58184bd3cce77bf",
              "IPY_MODEL_49d08e1666744788935d5cec78debec5"
            ],
            "layout": "IPY_MODEL_0b0c3bfa408944f0ab53ceacbb0290ab"
          }
        },
        "a0649663e0064c3898698e6ac70f8f0b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c65b1168a3844f3094282558ad6daec4",
            "placeholder": "​",
            "style": "IPY_MODEL_9f573622dba7461c80befb6147e821ed",
            "value": "#2: 100%"
          }
        },
        "e7fcf56d72ae4db2b58184bd3cce77bf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_942e27107b664be995ccaac6938b395f",
            "max": 2500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f11c5e703b8a41998f9f2e904d91ab97",
            "value": 2500
          }
        },
        "49d08e1666744788935d5cec78debec5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bfc4c27526ea463995d5ef9d25d8f87f",
            "placeholder": "​",
            "style": "IPY_MODEL_8eac35e6f76849de84986bd3888b9274",
            "value": " 2500/2500 [00:08&lt;00:00, 269.35ex/s]"
          }
        },
        "0b0c3bfa408944f0ab53ceacbb0290ab": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c65b1168a3844f3094282558ad6daec4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9f573622dba7461c80befb6147e821ed": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "942e27107b664be995ccaac6938b395f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f11c5e703b8a41998f9f2e904d91ab97": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "bfc4c27526ea463995d5ef9d25d8f87f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8eac35e6f76849de84986bd3888b9274": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d5b87682b53442bd8659eb783c7014e6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0a4beea1de764511995885126c16dcfe",
              "IPY_MODEL_024a4cc1b48944618993bdea6b4985b7",
              "IPY_MODEL_35f1a260b5b14a8eb1f22a590f681b55"
            ],
            "layout": "IPY_MODEL_8d904142c55740179553116f7035ae73"
          }
        },
        "0a4beea1de764511995885126c16dcfe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cae17ca9e5fe42ab8939059cb532d961",
            "placeholder": "​",
            "style": "IPY_MODEL_28893a1b202b492b9dda6374c7bd166f",
            "value": "100%"
          }
        },
        "024a4cc1b48944618993bdea6b4985b7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_518917b0fec24c248d26f8dfd8f1eafd",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_826159a8ace64488830020c7adccfa9f",
            "value": 1
          }
        },
        "35f1a260b5b14a8eb1f22a590f681b55": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_39c9cd1be375485e8f0ee7928485c715",
            "placeholder": "​",
            "style": "IPY_MODEL_7f047cb9d0e344bf9d4935fdc65fda3e",
            "value": " 1/1 [00:00&lt;00:00, 10.62ba/s]"
          }
        },
        "8d904142c55740179553116f7035ae73": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cae17ca9e5fe42ab8939059cb532d961": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "28893a1b202b492b9dda6374c7bd166f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "518917b0fec24c248d26f8dfd8f1eafd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "826159a8ace64488830020c7adccfa9f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "39c9cd1be375485e8f0ee7928485c715": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7f047cb9d0e344bf9d4935fdc65fda3e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8df570109ae44ed58643d60fbf28d992": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_88b1624e344846ec9ef0e56c15481aa8",
              "IPY_MODEL_cd1c493f3e9e475f93bfcaa2682824e0",
              "IPY_MODEL_ac5da44bdb2d44f6b3732ac2f8ad2f23"
            ],
            "layout": "IPY_MODEL_cb53dccd1d2b4912808ab5a849d7cbac"
          }
        },
        "88b1624e344846ec9ef0e56c15481aa8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c81fbb575fd8489eb62c1af4a859135e",
            "placeholder": "​",
            "style": "IPY_MODEL_fae5436ebc9b4b5c9cc34c16d6892436",
            "value": "100%"
          }
        },
        "cd1c493f3e9e475f93bfcaa2682824e0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b8d6535079f040d6b966cd824de41de2",
            "max": 11,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0413c1aeb87947769960174b8ccf0b1f",
            "value": 11
          }
        },
        "ac5da44bdb2d44f6b3732ac2f8ad2f23": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3fe76680764a40b0a2b6467130404f52",
            "placeholder": "​",
            "style": "IPY_MODEL_3ff0bc02b41449dbaa8ac38158908dd8",
            "value": " 11/11 [00:06&lt;00:00,  1.77ba/s]"
          }
        },
        "cb53dccd1d2b4912808ab5a849d7cbac": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c81fbb575fd8489eb62c1af4a859135e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fae5436ebc9b4b5c9cc34c16d6892436": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b8d6535079f040d6b966cd824de41de2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0413c1aeb87947769960174b8ccf0b1f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3fe76680764a40b0a2b6467130404f52": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3ff0bc02b41449dbaa8ac38158908dd8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4826a263fadb42bb96a655b9db53133a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2d1bf194f11f4011afa013b71930ef4b",
              "IPY_MODEL_5a62ea31f0ca405a9773dec50f65db47",
              "IPY_MODEL_aca809a58df14edf826dee6a9cb1f770"
            ],
            "layout": "IPY_MODEL_deae9b0000bd4b739bfa111d7834565f"
          }
        },
        "2d1bf194f11f4011afa013b71930ef4b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4ea66b71896e41eab8b36938a7987418",
            "placeholder": "​",
            "style": "IPY_MODEL_f4c6473c781d487cb45e835089332fb5",
            "value": "Upload file pytorch_model.bin: 100%"
          }
        },
        "5a62ea31f0ca405a9773dec50f65db47": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_31ce5ec915554de094c607d469b18e69",
            "max": 242070267,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_751e197802c448dfad695450ff65f3a7",
            "value": 242070267
          }
        },
        "aca809a58df14edf826dee6a9cb1f770": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_28d06f262040484aa3499de9a309ba98",
            "placeholder": "​",
            "style": "IPY_MODEL_04ba59fd0ff542cfb0d805e8ed90ee30",
            "value": " 231M/231M [03:16&lt;00:00, 1.28MB/s]"
          }
        },
        "deae9b0000bd4b739bfa111d7834565f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4ea66b71896e41eab8b36938a7987418": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f4c6473c781d487cb45e835089332fb5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "31ce5ec915554de094c607d469b18e69": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "751e197802c448dfad695450ff65f3a7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "28d06f262040484aa3499de9a309ba98": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "04ba59fd0ff542cfb0d805e8ed90ee30": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "69f4b04998564473b9d71d6f4bfe5c1a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_97e3906c0cba40bea907c6f6ce883be2",
              "IPY_MODEL_e38105c3925f4361bc23eb5f5844c8f9",
              "IPY_MODEL_5fe771b0963c419da78d4ed1aa2e73e4"
            ],
            "layout": "IPY_MODEL_79937bf895cc4968aa9d558777f0f44c"
          }
        },
        "97e3906c0cba40bea907c6f6ce883be2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a4ec4a140a0a4c17a251616521b3ff64",
            "placeholder": "​",
            "style": "IPY_MODEL_916f88b5ed9d4986a649be11c7e8c125",
            "value": "Upload file runs/May24_15-32-58_51560c61d591/events.out.tfevents.1653406576.51560c61d591.388.0: 100%"
          }
        },
        "e38105c3925f4361bc23eb5f5844c8f9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3eab5328f2664eb9b75639a4913a5df0",
            "max": 14830,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e3e7cafe1da64bcf98b45eda574f511e",
            "value": 14830
          }
        },
        "5fe771b0963c419da78d4ed1aa2e73e4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f81f143311c346eeb5b82f2aae381a88",
            "placeholder": "​",
            "style": "IPY_MODEL_23a71ee7af8c4f3191a9246f0935e99f",
            "value": " 14.5k/14.5k [03:14&lt;00:00, 58.5B/s]"
          }
        },
        "79937bf895cc4968aa9d558777f0f44c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a4ec4a140a0a4c17a251616521b3ff64": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "916f88b5ed9d4986a649be11c7e8c125": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3eab5328f2664eb9b75639a4913a5df0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e3e7cafe1da64bcf98b45eda574f511e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f81f143311c346eeb5b82f2aae381a88": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "23a71ee7af8c4f3191a9246f0935e99f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d35997d8d2f948b5a0ef51c7ef2445bc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c6aeb8f748e54ddda72e2b9695cc3917",
              "IPY_MODEL_aae50c2c22414f7ba20fc5b012be711c",
              "IPY_MODEL_805b522b877c4e798cb4338fb81be4de"
            ],
            "layout": "IPY_MODEL_00b87471cc52424791d85f1f6a6f1559"
          }
        },
        "c6aeb8f748e54ddda72e2b9695cc3917": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f9e7711606a044ebb8436c9592c9dde9",
            "placeholder": "​",
            "style": "IPY_MODEL_b1561b1d8de6445cbbeb9570941fde7a",
            "value": "Upload file runs/May24_15-32-58_51560c61d591/events.out.tfevents.1653412603.51560c61d591.388.2: 100%"
          }
        },
        "aae50c2c22414f7ba20fc5b012be711c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b7b7bee99d60493f8f5e2e86285586c4",
            "max": 495,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_80b30b9d7a5b40fc9260ce56936a718f",
            "value": 495
          }
        },
        "805b522b877c4e798cb4338fb81be4de": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ae9371e4642f4167a0895d6929cd303a",
            "placeholder": "​",
            "style": "IPY_MODEL_1eb8e84a09d642a5a891385a7c451e83",
            "value": " 495/495 [03:14&lt;?, ?B/s]"
          }
        },
        "00b87471cc52424791d85f1f6a6f1559": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f9e7711606a044ebb8436c9592c9dde9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b1561b1d8de6445cbbeb9570941fde7a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b7b7bee99d60493f8f5e2e86285586c4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "80b30b9d7a5b40fc9260ce56936a718f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ae9371e4642f4167a0895d6929cd303a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1eb8e84a09d642a5a891385a7c451e83": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}