{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prepare filelists for LJSpeech dataset\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# See: https://github.com/espeak-ng/espeak-ng/blob/master/docs/languages.md\n",
    "dir_data = \"/path/to/LJSpeech-1.1\"\n",
    "config = \"../config.yaml\"\n",
    "symlink = \"DUMMY1\"\n",
    "n_val = 100\n",
    "n_test = 500"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get hyperparameters from config file\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from utils.hparams import get_hparams_from_file\n",
    "\n",
    "hps = get_hparams_from_file(config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Read dataset\n",
    "\n",
    "Here `normalized_text` contains numbers in the form of words.\n",
    "\n",
    "**Note**: you may need to replace all `\"|\"` with `\" | \"` in the file `metadata.csv`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>file</th>\n",
       "      <th>text</th>\n",
       "      <th>normalized_text</th>\n",
       "      <th>cleaned_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DUMMY1/LJ001-0001.wav</td>\n",
       "      <td>Printing, in the only sense with which we are ...</td>\n",
       "      <td>Printing, in the only sense with which we are ...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>DUMMY1/LJ001-0002.wav</td>\n",
       "      <td>in being comparatively modern.</td>\n",
       "      <td>in being comparatively modern.</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>DUMMY1/LJ001-0003.wav</td>\n",
       "      <td>For although the Chinese took impressions from...</td>\n",
       "      <td>For although the Chinese took impressions from...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>DUMMY1/LJ001-0004.wav</td>\n",
       "      <td>produced the block books, which were the immed...</td>\n",
       "      <td>produced the block books, which were the immed...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DUMMY1/LJ001-0005.wav</td>\n",
       "      <td>the invention of movable metal letters in the ...</td>\n",
       "      <td>the invention of movable metal letters in the ...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    file                                               text  \\\n",
       "0  DUMMY1/LJ001-0001.wav  Printing, in the only sense with which we are ...   \n",
       "1  DUMMY1/LJ001-0002.wav                     in being comparatively modern.   \n",
       "2  DUMMY1/LJ001-0003.wav  For although the Chinese took impressions from...   \n",
       "3  DUMMY1/LJ001-0004.wav  produced the block books, which were the immed...   \n",
       "4  DUMMY1/LJ001-0005.wav  the invention of movable metal letters in the ...   \n",
       "\n",
       "                                     normalized_text  cleaned_text  \n",
       "0  Printing, in the only sense with which we are ...           NaN  \n",
       "1                     in being comparatively modern.           NaN  \n",
       "2  For although the Chinese took impressions from...           NaN  \n",
       "3  produced the block books, which were the immed...           NaN  \n",
       "4  the invention of movable metal letters in the ...           NaN  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv(\n",
    "    f\"{dir_data}/metadata_copy.csv\",\n",
    "    sep=r\"|\",\n",
    "    header=None,\n",
    "    names=[\"file\", \"text\", \"normalized_text\", \"cleaned_text\"],\n",
    "    index_col=False,\n",
    "    # converter to add .wav to file name\n",
    "    converters={\"file\": lambda x: f\"{symlink}/{x.strip()}.wav\", \"text\": str.strip, \"normalized_text\": str.strip},\n",
    ")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Text cleaners\n",
    "\n",
    "It may take a while, so better to preprocess the text and save it to a file in advance.\n",
    "\n",
    "**Note** `phonemize_text` takes the longest time.`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['tokenize_text', 'add_bos_eos']\n",
      "[['phonemize_text'], ['add_spaces']]\n"
     ]
    }
   ],
   "source": [
    "# Get index of tokenize_text\n",
    "text_cleaners = hps.data.text_cleaners\n",
    "\n",
    "token_idx = text_cleaners.index(\"tokenize_text\")\n",
    "token_cleaners = text_cleaners[token_idx:]\n",
    "print(token_cleaners)\n",
    "\n",
    "\n",
    "# Extract phonemize_text\n",
    "def separate_text_cleaners(text_cleaners):\n",
    "    final_list = []\n",
    "    temp_list = []\n",
    "\n",
    "    for cleaner in text_cleaners:\n",
    "        if cleaner == \"phonemize_text\":\n",
    "            if temp_list:\n",
    "                final_list.append(temp_list)\n",
    "            final_list.append([cleaner])\n",
    "            temp_list = []\n",
    "        else:\n",
    "            temp_list.append(cleaner)\n",
    "\n",
    "    if temp_list:\n",
    "        final_list.append(temp_list)\n",
    "\n",
    "    return final_list\n",
    "\n",
    "\n",
    "text_cleaners = text_cleaners[:token_idx]\n",
    "text_cleaners = separate_text_cleaners(text_cleaners)\n",
    "print(text_cleaners)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cleaning with ['phonemize_text'] ...\n",
      "Cleaning with ['add_spaces'] ...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>file</th>\n",
       "      <th>text</th>\n",
       "      <th>normalized_text</th>\n",
       "      <th>cleaned_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DUMMY1/LJ001-0001.wav</td>\n",
       "      <td>Printing, in the only sense with which we are ...</td>\n",
       "      <td>Printing, in the only sense with which we are ...</td>\n",
       "      <td>p ɹ ˈɪ n t ɪ ŋ , &lt;space&gt; ˈɪ n &lt;space&gt; ð ə &lt;spa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>DUMMY1/LJ001-0002.wav</td>\n",
       "      <td>in being comparatively modern.</td>\n",
       "      <td>in being comparatively modern.</td>\n",
       "      <td>ˈɪ n &lt;space&gt; b ˈiː ɪ ŋ &lt;space&gt; k ə m p ˈæ ɹ ə ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>DUMMY1/LJ001-0003.wav</td>\n",
       "      <td>For although the Chinese took impressions from...</td>\n",
       "      <td>For although the Chinese took impressions from...</td>\n",
       "      <td>f ɔːɹ &lt;space&gt; ɔː l ð ˈoʊ &lt;space&gt; ð ə &lt;space&gt; t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>DUMMY1/LJ001-0004.wav</td>\n",
       "      <td>produced the block books, which were the immed...</td>\n",
       "      <td>produced the block books, which were the immed...</td>\n",
       "      <td>p ɹ ə d ˈuː s t &lt;space&gt; ð ə &lt;space&gt; b l ˈɑː k ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DUMMY1/LJ001-0005.wav</td>\n",
       "      <td>the invention of movable metal letters in the ...</td>\n",
       "      <td>the invention of movable metal letters in the ...</td>\n",
       "      <td>ð ə &lt;space&gt; ɪ n v ˈɛ n ʃ ə n &lt;space&gt; ʌ v &lt;spac...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    file                                               text  \\\n",
       "0  DUMMY1/LJ001-0001.wav  Printing, in the only sense with which we are ...   \n",
       "1  DUMMY1/LJ001-0002.wav                     in being comparatively modern.   \n",
       "2  DUMMY1/LJ001-0003.wav  For although the Chinese took impressions from...   \n",
       "3  DUMMY1/LJ001-0004.wav  produced the block books, which were the immed...   \n",
       "4  DUMMY1/LJ001-0005.wav  the invention of movable metal letters in the ...   \n",
       "\n",
       "                                     normalized_text  \\\n",
       "0  Printing, in the only sense with which we are ...   \n",
       "1                     in being comparatively modern.   \n",
       "2  For although the Chinese took impressions from...   \n",
       "3  produced the block books, which were the immed...   \n",
       "4  the invention of movable metal letters in the ...   \n",
       "\n",
       "                                        cleaned_text  \n",
       "0  p ɹ ˈɪ n t ɪ ŋ , <space> ˈɪ n <space> ð ə <spa...  \n",
       "1  ˈɪ n <space> b ˈiː ɪ ŋ <space> k ə m p ˈæ ɹ ə ...  \n",
       "2  f ɔːɹ <space> ɔː l ð ˈoʊ <space> ð ə <space> t...  \n",
       "3  p ɹ ə d ˈuː s t <space> ð ə <space> b l ˈɑː k ...  \n",
       "4  ð ə <space> ɪ n v ˈɛ n ʃ ə n <space> ʌ v <spac...  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from text import tokenizer\n",
    "from torchtext.vocab import Vocab\n",
    "\n",
    "text_norm = data[\"normalized_text\"].tolist()\n",
    "for cleaners in text_cleaners:\n",
    "    print(f\"Cleaning with {cleaners} ...\")\n",
    "    if cleaners[0] == \"phonemize_text\":\n",
    "        text_norm = tokenizer(text_norm, Vocab, cleaners, language=hps.data.language)\n",
    "    else:\n",
    "        for idx, text in enumerate(text_norm):\n",
    "            temp = tokenizer(text, Vocab, cleaners, language=hps.data.language)\n",
    "            text_norm[idx] = temp\n",
    "\n",
    "data = data.assign(cleaned_text=text_norm)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate and save vocabulary\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of vocabulary: 129\n",
      "['<pad>', '<unk>', '<bos>', '<eos>', '<space>', '<laugh>', 'n', 't', 'ə', 's', 'd', 'ð', 'ɹ', 'k', 'z', 'ɪ', 'l', 'm', 'ˈɪ', 'p', 'w', 'v', 'ˈɛ', 'f', 'ˈeɪ', 'b', 'ɚ', ',', 'ʌ', 'ˈæ', 'h', 'ᵻ', 'i', 'æ', '.', 'ˈaɪ', 'ˈiː', 'ʃ', 'uː', 'ˈoʊ', 'ˈɑː', 'ˈʌ', 'ŋ', 'əl', 'ˈuː', 'ɾ', 'ɡ', 'ɐ', 'ˈɜː', 'dʒ', 'tʃ', 'iː', 'j', 'ˈaʊ', 'θ', 'ˌɪ', 'ˈɔː', 'ˈɔ', 'ˈoːɹ', 'ɔːɹ', 'ɛ', 'ˌɛ', 'ˌʌ', 'ˈɑːɹ', 'ˌæ', 'ˈɔːɹ', 'ˈʊ', 'ɜː', 'oʊ', 'eɪ', 'ˈɛɹ', 'ˈɪɹ', '\"', 'ˌeɪ', 'iə', 'ʊ', 'ˌaɪ', 'ˈɔɪ', 'ˌɑː', ';', 'aɪ', 'ɛɹ', 'ˈʊɹ', 'ɑːɹ', 'ʒ', 'ˈaɪɚ', 'ˌiː', 'ˌuː', 'ˌoʊ', 'aʊ', 'ˈiə', 'ɑː', 'ɔː', 'n̩', 'ʔ', 'ˈaɪə', ':', 'oːɹ', 'ˌaʊ', 'ˌɑːɹ', 'ˌɜː', 'ˌoː', 'ˈoː', '?', 'ˌɔːɹ', 'ˌɔː', 'ɪɹ', 'ʊɹ', 'oː', '!', 'ɔɪ', 'ˌʊɹ', 'ˌʊ', 'ˌiə', 'ˌɔɪ', 'r', 'ɔ', 'ˌoːɹ', 'aɪə', 'ˌɪɹ', 'aɪɚ', 'ˌɔ', 'ˌɛɹ', 'x', '“', '”', 'ˈɚ', 'ˌaɪɚ', 'ˌn̩']\n"
     ]
    }
   ],
   "source": [
    "from torchtext.vocab import build_vocab_from_iterator\n",
    "from utils.task import load_vocab, save_vocab\n",
    "from text.symbols import special_symbols, UNK_ID\n",
    "from typing import List\n",
    "\n",
    "\n",
    "def yield_tokens(cleaned_text: List[str]):\n",
    "    for text in cleaned_text:\n",
    "        yield text.split()\n",
    "\n",
    "\n",
    "text_norm = data[\"cleaned_text\"].tolist()\n",
    "vocab = build_vocab_from_iterator(yield_tokens(text_norm), specials=special_symbols)\n",
    "vocab.set_default_index(UNK_ID)\n",
    "\n",
    "vocab_file = f\"../vocab.txt\"\n",
    "save_vocab(vocab, vocab_file)\n",
    "\n",
    "vocab = load_vocab(vocab_file)\n",
    "print(f\"Size of vocabulary: {len(vocab)}\")\n",
    "print(vocab.get_itos())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Token cleaners\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>file</th>\n",
       "      <th>text</th>\n",
       "      <th>normalized_text</th>\n",
       "      <th>cleaned_text</th>\n",
       "      <th>tokens</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DUMMY1/LJ001-0001.wav</td>\n",
       "      <td>Printing, in the only sense with which we are ...</td>\n",
       "      <td>Printing, in the only sense with which we are ...</td>\n",
       "      <td>p ɹ ˈɪ n t ɪ ŋ , &lt;space&gt; ˈɪ n &lt;space&gt; ð ə &lt;spa...</td>\n",
       "      <td>2\\t19\\t12\\t18\\t6\\t7\\t15\\t42\\t27\\t4\\t18\\t6\\t4\\t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>DUMMY1/LJ001-0002.wav</td>\n",
       "      <td>in being comparatively modern.</td>\n",
       "      <td>in being comparatively modern.</td>\n",
       "      <td>ˈɪ n &lt;space&gt; b ˈiː ɪ ŋ &lt;space&gt; k ə m p ˈæ ɹ ə ...</td>\n",
       "      <td>2\\t18\\t6\\t4\\t25\\t36\\t15\\t42\\t4\\t13\\t8\\t17\\t19\\...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>DUMMY1/LJ001-0003.wav</td>\n",
       "      <td>For although the Chinese took impressions from...</td>\n",
       "      <td>For although the Chinese took impressions from...</td>\n",
       "      <td>f ɔːɹ &lt;space&gt; ɔː l ð ˈoʊ &lt;space&gt; ð ə &lt;space&gt; t...</td>\n",
       "      <td>2\\t23\\t59\\t4\\t92\\t16\\t11\\t39\\t4\\t11\\t8\\t4\\t50\\...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>DUMMY1/LJ001-0004.wav</td>\n",
       "      <td>produced the block books, which were the immed...</td>\n",
       "      <td>produced the block books, which were the immed...</td>\n",
       "      <td>p ɹ ə d ˈuː s t &lt;space&gt; ð ə &lt;space&gt; b l ˈɑː k ...</td>\n",
       "      <td>2\\t19\\t12\\t8\\t10\\t44\\t9\\t7\\t4\\t11\\t8\\t4\\t25\\t1...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DUMMY1/LJ001-0005.wav</td>\n",
       "      <td>the invention of movable metal letters in the ...</td>\n",
       "      <td>the invention of movable metal letters in the ...</td>\n",
       "      <td>ð ə &lt;space&gt; ɪ n v ˈɛ n ʃ ə n &lt;space&gt; ʌ v &lt;spac...</td>\n",
       "      <td>2\\t11\\t8\\t4\\t15\\t6\\t21\\t22\\t6\\t37\\t8\\t6\\t4\\t28...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    file                                               text  \\\n",
       "0  DUMMY1/LJ001-0001.wav  Printing, in the only sense with which we are ...   \n",
       "1  DUMMY1/LJ001-0002.wav                     in being comparatively modern.   \n",
       "2  DUMMY1/LJ001-0003.wav  For although the Chinese took impressions from...   \n",
       "3  DUMMY1/LJ001-0004.wav  produced the block books, which were the immed...   \n",
       "4  DUMMY1/LJ001-0005.wav  the invention of movable metal letters in the ...   \n",
       "\n",
       "                                     normalized_text  \\\n",
       "0  Printing, in the only sense with which we are ...   \n",
       "1                     in being comparatively modern.   \n",
       "2  For although the Chinese took impressions from...   \n",
       "3  produced the block books, which were the immed...   \n",
       "4  the invention of movable metal letters in the ...   \n",
       "\n",
       "                                        cleaned_text  \\\n",
       "0  p ɹ ˈɪ n t ɪ ŋ , <space> ˈɪ n <space> ð ə <spa...   \n",
       "1  ˈɪ n <space> b ˈiː ɪ ŋ <space> k ə m p ˈæ ɹ ə ...   \n",
       "2  f ɔːɹ <space> ɔː l ð ˈoʊ <space> ð ə <space> t...   \n",
       "3  p ɹ ə d ˈuː s t <space> ð ə <space> b l ˈɑː k ...   \n",
       "4  ð ə <space> ɪ n v ˈɛ n ʃ ə n <space> ʌ v <spac...   \n",
       "\n",
       "                                              tokens  \n",
       "0  2\\t19\\t12\\t18\\t6\\t7\\t15\\t42\\t27\\t4\\t18\\t6\\t4\\t...  \n",
       "1  2\\t18\\t6\\t4\\t25\\t36\\t15\\t42\\t4\\t13\\t8\\t17\\t19\\...  \n",
       "2  2\\t23\\t59\\t4\\t92\\t16\\t11\\t39\\t4\\t11\\t8\\t4\\t50\\...  \n",
       "3  2\\t19\\t12\\t8\\t10\\t44\\t9\\t7\\t4\\t11\\t8\\t4\\t25\\t1...  \n",
       "4  2\\t11\\t8\\t4\\t15\\t6\\t21\\t22\\t6\\t37\\t8\\t6\\t4\\t28...  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from text import detokenizer\n",
    "\n",
    "text_norm = data[\"cleaned_text\"].tolist()\n",
    "for idx, text in enumerate(text_norm):\n",
    "    temp = tokenizer(text, vocab, token_cleaners, language=hps.data.language)\n",
    "    assert UNK_ID not in temp, f\"Found unknown symbol:\\n{text}\\n{detokenizer(temp)}\"\n",
    "    text_norm[idx] = temp\n",
    "\n",
    "text_norm = [\"\\t\".join(map(str, text)) for text in text_norm]\n",
    "data = data.assign(tokens=text_norm)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save train, val, test filelists\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = data[[\"file\", \"tokens\"]]\n",
    "data = data.sample(frac=1).reset_index(drop=True)\n",
    "\n",
    "data_train = data.iloc[n_val + n_test:]\n",
    "data_val = data.iloc[:n_val]\n",
    "data_test = data.iloc[n_val: n_val + n_test]\n",
    "\n",
    "data_train.to_csv(\"../filelists/train.txt\", sep=\"|\", index=False, header=False)\n",
    "data_val.to_csv(\"../filelists/val.txt\", sep=\"|\", index=False, header=False)\n",
    "data_test.to_csv(\"../filelists/test.txt\", sep=\"|\", index=False, header=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "naturalspeech",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
