{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "YOLACT++.ipynb",
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mrm8488/shared_colab_notebooks/blob/master/YOLACT%2B%2B.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HO0JwO9csRAk",
        "colab_type": "text"
      },
      "source": [
        "# **Y**ou **O**nly **L**ook **A**t **C**oefficien**T**s\n",
        "```\n",
        "    ██╗   ██╗ ██████╗ ██╗      █████╗  ██████╗████████╗\n",
        "    ╚██╗ ██╔╝██╔═══██╗██║     ██╔══██╗██╔════╝╚══██╔══╝\n",
        "     ╚████╔╝ ██║   ██║██║     ███████║██║        ██║   \n",
        "      ╚██╔╝  ██║   ██║██║     ██╔══██║██║        ██║   \n",
        "       ██║   ╚██████╔╝███████╗██║  ██║╚██████╗   ██║   \n",
        "       ╚═╝    ╚═════╝ ╚══════╝╚═╝  ╚═╝ ╚═════╝   ╚═╝ \n",
        "```\n",
        "\n",
        "A simple, fully convolutional model for real-time instance segmentation. This is the code for our papers:\n",
        " - [YOLACT: Real-time Instance Segmentation](https://arxiv.org/abs/1904.02689)\n",
        " - [YOLACT++: Better Real-time Instance Segmentation](https://arxiv.org/abs/1912.06218)\n",
        "\n",
        "#### YOLACT++ (v1.2) released! ([Changelog](CHANGELOG.md))\n",
        "YOLACT++'s resnet50 model runs at 33.5 fps on a Titan Xp and achieves 34.1 mAP on COCO's `test-dev` (check out our journal paper [here](https://arxiv.org/abs/1912.06218)).\n",
        "\n",
        "[GitHub](https://github.com/dbolya/yolact)\n",
        "\n",
        "> Colab author: [mrm8488](https://twitter.com/mrm8488)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aVJsG_GUs1QZ",
        "colab_type": "text"
      },
      "source": [
        "# Install required packages"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l1naq-OprvLQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Cython needs to be installed before pycocotools\n",
        "!pip install cython\n",
        "!pip install opencv-python pillow pycocotools matplotlib "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p9iG8ub89xRX",
        "colab_type": "text"
      },
      "source": [
        "# Clone the repository"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rjkNTFlMtC2i",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 141
        },
        "outputId": "6eac3b02-3eaf-49a4-976c-3c2d43bd8a8c"
      },
      "source": [
        "!git clone https://github.com/dbolya/yolact.git"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'yolact'...\n",
            "remote: Enumerating objects: 20, done.\u001b[K\n",
            "remote: Counting objects: 100% (20/20), done.\u001b[K\n",
            "remote: Compressing objects: 100% (18/18), done.\u001b[K\n",
            "remote: Total 2858 (delta 0), reused 7 (delta 0), pack-reused 2838\u001b[K\n",
            "Receiving objects: 100% (2858/2858), 21.20 MiB | 15.44 MiB/s, done.\n",
            "Resolving deltas: 100% (1942/1942), done.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kLMA2Ebxw5tT",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "99281b68-9b13-4ad8-ab8b-3ca220d01e93"
      },
      "source": [
        "cd yolact"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/yolact\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2tQjvVnCxQJ2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!mkdir weights"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i46hGwDh92VM",
        "colab_type": "text"
      },
      "source": [
        "# Download the model weights"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bXBoJjdYuNCB",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!gdown \"https://drive.google.com/uc?id=15id0Qq5eqRbkD-N3ZjDZXdCvRyIaHpFB&export=download\" -O \"./weights/yolact_plus_base_54_800000.pth\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SZt7kNerxgOJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "cd external/DCNv2"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mu-UK-Of-N8F",
        "colab_type": "text"
      },
      "source": [
        "# Compile deformable convolutional layers (from DCNv2) to use YOLACT++ version"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HDrNO2vIxh05",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!python setup.py build develop &> /dev/null"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rQgYBYmtx-Q-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "5e634356-71e9-4ab8-cea0-daed9f7a13f4"
      },
      "source": [
        "cd /content/yolact/"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/yolact\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ATZ5d6B83ptv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!mkdir uploads\n",
        "!mkdir results"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p-TWRro498oW",
        "colab_type": "text"
      },
      "source": [
        "# Upload files from local filesystem"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Js12lLSb35P3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from google.colab import files\n",
        "\n",
        "uploaded = files.upload()\n",
        "pic_names = list(uploaded.keys())\n",
        "for pic_name in pic_names:\n",
        "  print(pic_name)\n",
        "  !mv ./$pic_name ./uploads/$pic_name"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dkank5TW-XgL",
        "colab_type": "text"
      },
      "source": [
        "# Detect objects in the uploaded pictures"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kJKl9ajI6kzM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!python eval.py --trained_model=weights/yolact_plus_base_54_800000.pth --score_threshold=0.15 --top_k=15 --images=./uploads:./results"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PtXCvKtl-cfZ",
        "colab_type": "text"
      },
      "source": [
        "# Display results"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a5oI6Ewb2Ooy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from IPython.display import display, Image\n",
        "import os\n",
        "\n",
        "for filename in os.listdir(\"./results/\"):\n",
        "    print(filename)\n",
        "    if filename.endswith(\".png\"):\n",
        "      img = Image(\"./results/\"+filename)\n",
        "      display(img)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K1GCG-Mk-iZk",
        "colab_type": "text"
      },
      "source": [
        "# Zip and download the results"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "69_iY-TU9RtD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!zip -r /content/results_images.zip ./results"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vmt-dMDo9qgh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from google.colab import files\n",
        "files.download(\"/content/results_images.zip\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bOHemwQG-w9k",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}