{"pred": "Unanswerable", "answers": ["Ground truth is not established in the paper"], "all_classes": null, "length": 3141}
{"pred": "GhostVLAD approach is an extension of the NetVLAD approach, which was originally proposed for face recognition. It adds Ghost clusters along with the NetVLAD clusters, where the Ghost clusters are used to map noisy or irrelevant content into ghost clusters and are excluded during feature aggregation.", "answers": ["extension of the NetVLAD, adds Ghost clusters along with the NetVLAD clusters", "An extension of NetVLAD which replaces hard assignment-based clustering with soft assignment-based clustering with the additon o fusing Ghost clusters to deal with noisy content."], "all_classes": null, "length": 2454}
{"pred": "Yes", "answers": ["the attention model, MDREA, also outperforms the best existing research results (WAP 0.690 to 0.688)"], "all_classes": null, "length": 3207}
{"pred": "The article proposes using context tweets as additional features for neural network models. Context tweets are defined as tweets that the user has replied to or quoted. The authors concatenate the max-pooled layers of context and labeled tweets for the CNN baseline model, and the last hidden states of context and labeled tweets for the RNN baseline model.", "answers": ["using tweets that one has replied or quoted to as contextual information", "text sequences of context tweets"], "all_classes": null, "length": 2060}
{"pred": "Facebook pages like FoxNews, CNN, ESPN, New York Times, Time magazine, Huffington Post Weird News, The Guardian, Cartoon Network, Cooking Light, Home Cooking Adventure, Justin Bieber, Nickelodeon, Spongebob, Disney.", "answers": ["FoxNews, CNN, ESPN, New York Times, Time magazine, Huffington Post Weird News, The Guardian, Cartoon Network, Cooking Light, Home Cooking Adventure, Justin Bieber, Nickelodeon, Spongebob, Disney", "FoxNews, CNN, ESPN, New York Times, Time magazine, Huffington Post Weird News, The Guardian, Cartoon Network, Cooking Light, Home Cooking Adventure, Justin Bieber, Nickelodeon, Spongebob, Disney."], "all_classes": null, "length": 3411}
{"pred": "No", "answers": ["Yes", "Yes"], "all_classes": null, "length": 3735}
{"pred": "Concept-map-based MDS is evaluated using a baseline method and evaluation scripts that are released along with the corpus.", "answers": ["Answer with content missing: (Evaluation Metrics section) Precision, Recall, F1-scores, Strict match, METEOR, ROUGE-2"], "all_classes": null, "length": 4263}
{"pred": "CNN/DailyMail, NYT, XSum", "answers": ["CNN/DailyMail news highlights, New York Times Annotated Corpus, XSum", "the CNN/DailyMail news highlights dataset BIBREF24, the New York Times Annotated Corpus (NYT; BIBREF25), XSum BIBREF22"], "all_classes": null, "length": 4369}
{"pred": "Yes. The article states that the proposed approach achieves better correlation scores than existing word sense disambiguation (WSD) approaches that use word embeddings, as shown in the experimental results on benchmark datasets.", "answers": ["GM$\\_$KL achieves better correlation than existing approaches for various metrics on SCWS dataset."], "all_classes": null, "length": 2189}
{"pred": "The answer is unanswerable based on the information in the article. The article does not describe the specifics of their ensemble method or how they formed the final model.", "answers": ["simply averaging the predictions from the constituent single models"], "all_classes": null, "length": 4212}
{"pred": "Yes. The article mentions that the Friends dataset comes from the scripts of the Friends TV sitcom, while the EmotionPush dataset is composed of Facebook messenger chats.", "answers": ["Friends TV sitcom, Facebook messenger chats"], "all_classes": null, "length": 3185}
{"pred": "Yes", "answers": ["English", "Simple English"], "all_classes": null, "length": 2243}
{"pred": "Yes, the IMDb dataset is used for sentiment analysis.", "answers": ["IMDb dataset of movie reviews", "IMDb"], "all_classes": null, "length": 2327}
{"pred": "No answer can be found in the article. Therefore, the answer is \"unanswerable\".", "answers": ["F1 scores of 85.99 on the DL-PS data,  75.15 on the EC-MT data and 71.53 on the EC-UQ data ", "F1 of 85.99 on the DL-PS dataset (dialog domain); 75.15 on EC-MT and 71.53 on EC-UQ (e-commerce domain)"], "all_classes": null, "length": 5310}
{"pred": "Yes", "answers": ["No"], "all_classes": null, "length": 3445}
{"pred": "Unanswerable", "answers": ["Custom dataset with user questions; set of documents, twitter posts and news articles, all related to finance.", "a self-collected financial intents dataset in Portuguese"], "all_classes": null, "length": 13401}
{"pred": "Unanswerable", "answers": ["Energy with accuracy of 0.538", "Energy"], "all_classes": null, "length": 10349}
{"pred": "Yes. They compared the RNN-based NMT model with the Transformer-based NMT model.", "answers": ["RNN-based NMT model, Transformer-NMT"], "all_classes": null, "length": 3708}
{"pred": "The three regularization terms are:\n\n1. A regularization term associated with neutral features\n2. The maximum entropy of class distribution regularization term\n3. The KL divergence between reference and predicted class distribution", "answers": ["a regularization term associated with neutral features, the maximum entropy of class distribution regularization term, the KL divergence between reference and predicted class distribution", "a regularization term associated with neutral features,  the maximum entropy of class distribution, KL divergence between reference and predicted class distribution"], "all_classes": null, "length": 3604}
{"pred": "The baselines are:\n\n1. SVM with unigram, bigram, and trigram features\n\n2. SVM with average word embedding\n\n3. SVM with average transformed word embeddings\n\n4. CNN and Recurrent Convolutional Neural Networks (RCNN)\n\n5. SVM and deep learning models with comment information\n\n6. UTCNN without user information\n\n7. UTCNN without topic information\n\n8. UTCNN without comments", "answers": ["SVM with unigram, bigram, and trigram features, SVM with average word embedding, SVM with average transformed word embeddings, CNN, ecurrent Convolutional Neural Networks, SVM and deep learning models with comment information", "SVM with unigram, bigram, trigram features, with average word embedding, with average transformed word embeddings, CNN and RCNN, SVM, CNN, RCNN with comment information"], "all_classes": null, "length": 4512}
{"pred": "Yes", "answers": ["They decrease MAE in 0.34"], "all_classes": null, "length": 2735}
{"pred": "Yes. The article states that their adaptively sparse transformer model with entmax attention allows for more diverse and specialized attention heads, which can be analyzed to gain insights into the model's behavior and interpretability.", "answers": ["the attention heads in the proposed adaptively sparse Transformer can specialize more and with higher confidence", "We introduce sparse attention into the Transformer architecture"], "all_classes": null, "length": 4902}
{"pred": "The baseline was a sentence-level machine translation model.", "answers": [" MT system on the data released by BIBREF11", "Transformer base, two-pass CADec model"], "all_classes": null, "length": 3716}
{"pred": " Yes. The article mentions that they evaluate their models using zero-shot cross-lingual transfer tasks like XNLI and dependency parsing, as well as supervised dependency parsing tasks. They report accuracy scores and labeled attachment scores (LAS) as metrics.", "answers": ["translation probabilities, Labeled Attachment Scores (LAS)", "accuracy, Labeled Attachment Scores (LAS)"], "all_classes": null, "length": 3402}
{"pred": "Yes. The attention module is pretrained on the MT task.", "answers": ["the model is pre-trained on CTC-based ASR task and MT task in the pre-training stage."], "all_classes": null, "length": 4656}
{"pred": "Unanswerable", "answers": ["Unanswerable"], "all_classes": null, "length": 3543}
{"pred": "The encoder has an LSTM encoder.", "answers": ["LSTM", "LSTM"], "all_classes": null, "length": 2289}
{"pred": "Unanswerable", "answers": ["Unanswerable", "Yes"], "all_classes": null, "length": 6391}
{"pred": "Yes", "answers": ["Unanswerable", "LF-MMI Attention\nSeq2Seq \nRNN-T \nChar E2E LF-MMI \nPhone E2E LF-MMI \nCTC + Gram-CTC"], "all_classes": null, "length": 1856}
{"pred": "22,880 users\n\n", "answers": ["22,880 users", "20,000"], "all_classes": null, "length": 4160}
{"pred": "Yes. The article mentions that they evaluate their models using perplexity, BLEU-1/4, ROUGE-L, recipe-level coherence, and step entailment metrics.", "answers": ["Byte-Pair Encoding perplexity  (BPE PPL),\nBLEU-1,\nBLEU-4,\nROUGE-L,\npercentage of distinct unigram (D-1),\npercentage of distinct bigrams(D-2),\nuser matching accuracy(UMA),\nMean Reciprocal Rank(MRR)\nPairwise preference over baseline(PP)", "BLEU-1/4 and ROUGE-L, likelihood of generated recipes using identical input specifications but conditioned on ten different user profiles, user matching accuracy (UMA), Mean Reciprocal Rank (MRR), neural scoring model from BIBREF33 to measure recipe-level coherence", " Distinct-1/2, UMA = User Matching Accuracy, MRR\n= Mean Reciprocal Rank, PP = Pairwise preference over baseline (evaluated for 310 recipe pairs per model)"], "all_classes": null, "length": 2673}
{"pred": "Yes", "answers": ["(1) the time the patient has been experiencing the symptom, (2) activities that trigger the symptom (to occur or worsen), (3) the extent of seriousness, (4) the frequency occurrence of the symptom, and (5) the location of symptom, No Answer", "the time the patient has been experiencing the symptom, activities that trigger the symptom, the extent of seriousness, the frequency occurrence of the symptom, the location of symptom, 9 symptoms"], "all_classes": null, "length": 3424}
{"pred": "Unanswerable\n\nThe article does not provide information on how much data is needed to train the task-specific encoder.", "answers": ["57,505 sentences", "57,505 sentences"], "all_classes": null, "length": 4371}
{"pred": " Evaluation is done on four machine translation tasks.\n\n\"Yes\"", "answers": ["four machine translation tasks: German -> English, Japanese -> English, Romanian -> English, English -> German", " four machine translation tasks, IWSLT 2017 German $\\rightarrow $ English BIBREF27, KFTT Japanese $\\rightarrow $ English BIBREF28, WMT 2016 Romanian $\\rightarrow $ English BIBREF29, WMT 2014 English $\\rightarrow $ German BIBREF30"], "all_classes": null, "length": 4898}
{"pred": "No\n\nThe article does not provide any information about the improvement in performance for Estonian in the NER task. Therefore, the answer is \"unanswerable\".", "answers": ["5 percent points.", "0.05 F1"], "all_classes": null, "length": 3290}
{"pred": "Unanswerable", "answers": ["Unanswerable"], "all_classes": null, "length": 8506}
{"pred": "Yes", "answers": ["No", "No"], "all_classes": null, "length": 2239}
{"pred": "Unanswerable", "answers": ["Nguni languages (zul, xho, nbl, ssw), Sotho languages (nso, sot, tsn)", "The Nguni languages are similar to each other, The same is true of the Sotho languages"], "all_classes": null, "length": 1877}
{"pred": " They compared 6-layers and 9-layers sMBR models.", "answers": ["Unidirectional LSTM networks with 2, 6, 7, 8, and  9 layers."], "all_classes": null, "length": 4286}
{"pred": "Unanswerable", "answers": ["a sample of  29,794 wikipedia articles and 2,794 arXiv papers "], "all_classes": null, "length": 4187}
{"pred": "Unanswerable", "answers": ["50 human annotators ranked a random sample of 100 translations by Adequacy, Fluency and overall ranking on a 5-point scale.", "adequacy, precision and ranking values"], "all_classes": null, "length": 5344}
{"pred": "No", "answers": ["Yes", "Yes"], "all_classes": null, "length": 4472}
{"pred": "Yes. The models are evaluated based on their ability to reconstruct the target sequence $x$ from the keywords $z$ generated by the user. The authors measure the accuracy of the reconstruction by comparing the generated sequence to the target sequence, and the efficiency of the model by counting the number of tokens retained in the keywords.", "answers": ["by training an autocomplete system on 500K randomly sampled sentences from Yelp reviews", "efficiency of a communication scheme $(q_{\\alpha },p_{\\beta })$ by the retention rate of tokens, which is measured as the fraction of tokens that are kept in the keywords, accuracy of a scheme is measured as the fraction of sentences generated by greedily decoding the model that exactly matches the target sentence"], "all_classes": null, "length": 1873}
{"pred": "Precision, recall, F-score, ROUGE unigram score.", "answers": ["Precision, Recall, F-measure, accuracy", "Precision, Recall and F-measure"], "all_classes": null, "length": 3044}
{"pred": "Unanswerable", "answers": ["Book, electronics, beauty, music, IMDB, Yelp, cell phone, baby, DVDs, kitchen", "we use set 1 of the source domain as the only source with sentiment label information during training, and we evaluate the trained model on set 1 of the target domain, Book (BK), Electronics (E), Beauty (BT), and Music (M)"], "all_classes": null, "length": 5061}
{"pred": "The article compares the PRU with the following previous RNN models:\n\n- LSTM\n- RAN\n- QRNN\n- NAS\n\nSo the answer is \"yes\".", "answers": ["Variational LSTM, CharCNN, Pointer Sentinel-LSTM, RHN, NAS Cell, SRU, QRNN, RAN, 4-layer skip-connection LSTM, AWD-LSTM, Quantized LSTM"], "all_classes": null, "length": 3319}
{"pred": "NeuronBlocks includes the following neural network modules:\n\n- Word/character embedding\n- RNN\n- CNN\n- QRNN\n- Transformer\n- Attention mechanisms\n- Regularization layers\n\nUnanswerable", "answers": ["Embedding Layer, Neural Network Layers, Loss Function, Metrics", "Embedding Layer, Neural Network Layers, Loss Function, Metrics"], "all_classes": null, "length": 1678}
{"pred": "The article mentions the following datasets:\n\n- Carnegie Mellon Pronouncing Dictionary\n- multilingual pronunciation corpus collected by deri2016grapheme\n- Wiktionary\n\nSo the answer is:\n\nThe datasets used were the Carnegie Mellon Pronouncing Dictionary, the multilingual pronunciation corpus collected by deri2016grapheme, and Wiktionary.", "answers": ["the Carnegie Mellon Pronouncing Dictionary BIBREF12, the multilingual pronunciation corpus collected by deri2016grapheme , ranscriptions extracted from Wiktionary", "multilingual pronunciation corpus collected by deri2016grapheme"], "all_classes": null, "length": 3244}
{"pred": "Yes", "answers": ["varied from Maximum Entropy Classifiers (BIBREF4) to Support Vector Machines (BIBREF5,BIBREF6,BIBREF7,BIBREF8), Recursive Neural Networks (BIBREF9,BIBREF10), Convolutional Neural Networks (BIBREF11) and most recently transfer learning-based architectures like Bidirectional Encoder Representation from Transformers (BERT) (BIBREF12)"], "all_classes": null, "length": 2215}
{"pred": "English, Spanish, Finnish", "answers": ["English\nFrench\nSpanish\nGerman\nGreek\nBulgarian\nRussian\nTurkish\nArabic\nVietnamese\nThai\nChinese\nHindi\nSwahili\nUrdu\nFinnish", "English, Spanish, Finnish"], "all_classes": null, "length": 4086}
{"pred": "Yes", "answers": ["None"], "all_classes": null, "length": 2473}
{"pred": "No", "answers": ["Yes", "Yes"], "all_classes": null, "length": 4457}
{"pred": "Yes", "answers": ["No", "No"], "all_classes": null, "length": 2738}
{"pred": "Unanswerable", "answers": ["using the Meaning Extraction Method", "Unanswerable"], "all_classes": null, "length": 1440}
{"pred": "The ML methods aim to identify the following argument components: Claim-B, Claim-I, Backing-B, Backing-I, Premise-B, Premise-I, Rebuttal-B, Rebuttal-I, Refutation-B, Refutation-I.", "answers": ["claim, premise, backing, rebuttal, and refutation", "claim, premise, backing, rebuttal, refutation"], "all_classes": null, "length": 14472}
{"pred": "PARENT aligns n-grams of length INLINEFORM0.", "answers": ["Unanswerable", "Answer with content missing: (Parent subsections) combine precisions for n-gram orders 1-4"], "all_classes": null, "length": 3827}
{"pred": "Twitter dataset consists of 1,873 Twitter conversation threads, roughly 14k tweets.\n\n", "answers": ["1,873 Twitter conversation threads, roughly 14k tweets", "1,873 Twitter conversation threads, roughly 14k tweets"], "all_classes": null, "length": 3721}
{"pred": "Unanswerable", "answers": ["Chinese Mandarin, Welsh, English, Estonian, Finnish, French, Hebrew, Polish, Russian, Spanish, Kiswahili, Yue Chinese", "Chinese Mandarin, Welsh, English, Estonian, Finnish, French, Hebrew, Polish, Russian, Spanish, Kiswahili, Yue Chinese"], "all_classes": null, "length": 14660}
{"pred": "Wikipedia conversations and Reddit conversations.", "answers": [" `Conversations Gone Awry' dataset, subreddit ChangeMyView", "An expanded version of the existing 'Conversations Gone Awry' dataset and the ChangeMyView dataset, a subreddit whose only annotation is whether the conversation required action by the Reddit moderators. "], "all_classes": null, "length": 4718}
{"pred": "No", "answers": ["No", "No"], "all_classes": null, "length": 2276}
{"pred": "Yes. The article describes various methods to evaluate the quality of the data, including:\n\n- Sanity checks on translations to ensure they are of good quality\n- Calculating BLEU scores between human and automatic translations\n- Measuring perplexity of translations\n- Checking for overlap between transcripts and translations\n- Evaluating model performance on standard tasks like ASR and MT\n\nSo the data is empirically evaluated through these various methods.", "answers": ["Validated transcripts were sent to professional translators., various sanity checks to the translations,  sanity check the overlaps of train, development and test sets", "computed sentence-level BLEU, We manually inspected examples where the source transcript was identical to the translation, measured the perplexity of the translations, computed the ratio of English characters in the translations, calculate similarity scores between transcripts and translations"], "all_classes": null, "length": 2435}
{"pred": "The proposed model uses dual RNNs to encode audio and text sequences independently. The audio-RNN encodes MFCC features from the audio signal using equation EQREF2 . The last hidden state of the audio-RNN is concatenated with the prosodic features to form the final vector representation INLINEFORM0 , and this vector is then passed through a fully connected neural network layer to form the audio encoding vector A. On the other hand, the text-RNN encodes the word sequence of the transcript using equation EQREF2 . The final hidden states of the text-RNN are also passed", "answers": ["combines the information from these sources using a feed-forward neural model", "encodes the information from audio and text sequences using dual RNNs and then combines the information from these sources using a feed-forward neural model"], "all_classes": null, "length": 3201}
{"pred": "Yes", "answers": ["For the WikiLarge dataset, the improvement over baseline NMT is 2.11 BLEU, 1.7 FKGL and 1.07 SARI.\nFor the WikiSmall dataset, the improvement over baseline NMT is  8.37 BLEU.", "6.37 BLEU"], "all_classes": null, "length": 2271}
{"pred": "700 humans evaluated the results.", "answers": ["Unanswerable", "Unanswerable"], "all_classes": null, "length": 3711}
{"pred": "Yes", "answers": ["Viral tweets are the ones that are retweeted more than 1000 times", "those that contain a high number of retweets"], "all_classes": null, "length": 3144}
{"pred": "BERT", "answers": ["BERT"], "all_classes": null, "length": 1507}
{"pred": "The data collection project was mainly supported by Sharif DeepMine company.", "answers": ["Android application"], "all_classes": null, "length": 3795}
{"pred": "Deep learning model.", "answers": ["Logistic Regression, neural networks"], "all_classes": null, "length": 7257}
{"pred": "The Social Honeypot dataset is the benchmark dataset used in the paper. The paper states that it contains 19,276 legitimate users and 2,223 spammers collected over 7 months. The quality of the dataset is considered high.\n\n", "answers": ["Social Honeypot dataset (public) and Weibo dataset (self-collected); yes", "Social Honeypot, which is not of high quality"], "all_classes": null, "length": 2242}
{"pred": "The decoder has an LSTM encoder followed by an LSTM decoder.", "answers": ["LSTM", "LSTM"], "all_classes": null, "length": 2289}
{"pred": "No", "answers": ["Unanswerable", "Unanswerable"], "all_classes": null, "length": 3838}
{"pred": " Yes, the ensemble of Logistic Regression, CNN and BERT achieved the best performance among the author's submissions, with a F1 score of 0.673 on the external dev set for sentence-level propaganda detection.", "answers": ["For SLC task, the \"ltuorp\" team  has the best performing  model (0.6323/0.6028/0.6649 for F1/P/R  respectively) and for FLC task the \"newspeak\" team  has the best performing  model (0.2488/0.2863/0.2201 for F1/P/R respectively)."], "all_classes": null, "length": 1541}
{"pred": "The baseline was a uni-directional PBSMT model.", "answers": ["pivot-based translation relying on a helping language BIBREF10, nduction of phrase tables from monolingual data BIBREF14 , attentional RNN-based model (RNMT) BIBREF2, Transformer model BIBREF18, bi-directional model BIBREF11, multi-to-multi (M2M) model BIBREF8, back-translation BIBREF17", "M2M Transformer"], "all_classes": null, "length": 4542}
{"pred": " Yes, their highest recall score was 0.7033 for List-type questions.", "answers": ["0.7033", "0.7033"], "all_classes": null, "length": 6810}
{"pred": "Unanswerable", "answers": ["Skip–gram, CBOW", "integrated vector-res, vector-faith, Skip–gram, CBOW"], "all_classes": null, "length": 4259}
{"pred": "Yes", "answers": ["Unanswerable", "CFILT-preorder system"], "all_classes": null, "length": 2231}
{"pred": "Yes", "answers": ["Yes"], "all_classes": null, "length": 3035}
{"pred": "Experts with legal training were used for annotation.", "answers": ["Individuals with legal training", "Yes"], "all_classes": null, "length": 3846}
{"pred": " The CNN-RNN model is used for image embedding and the sequence-to-sequence model with parallel text corpus is used for language style transfer.", "answers": ["generating a poem from images we use an existing actor-critic architecture, various types of sequence to sequence models"], "all_classes": null, "length": 1653}
{"pred": "RoBERT\n\n", "answers": ["Transformer over BERT (ToBERT)", "The transformer layer"], "all_classes": null, "length": 2655}
{"pred": "No", "answers": ["Yes", "Yes"], "all_classes": null, "length": 4127}
{"pred": "Yes, they addressed three topics of cyberbullying: personal attack, racism, and sexism.", "answers": ["personal attack, racism, and sexism", "racism, sexism, personal attack, not specifically about any single topic"], "all_classes": null, "length": 3244}
{"pred": "The authors propose a new context representation for CNNs called \"extended middle context\" which uses all parts of the sentence (the relation arguments, left of the relation arguments, between the arguments, and right of the arguments) and pays special attention to the middle part. They split the context into two parts: a combination of the left context, the left entity and the middle context; and a combination of the middle context, the right entity and the right context.", "answers": ["They use two independent convolutional and max-pooling layers on (1) a combination of the left context, the left entity and the middle context; and (2) a combination of the middle context, the right entity and the right context. They concatenated the two results after pooling to get the new context representation."], "all_classes": null, "length": 2435}
{"pred": "Yes. The paper mentions that the dataset contains three major categories of entities: Person (PER), Location (LOC), and Organization (ORG).", "answers": ["OurNepali contains 3 different types of entities, ILPRL contains 4 different types of entities", "three"], "all_classes": null, "length": 2851}
{"pred": "The article does not provide information on how much higher quality the resulting annotated data is. Therefore, the answer is \"unanswerable\".", "answers": ["improvement when the difficult subset with expert annotations is mixed with the remaining crowd annotation is 3.5 F1 score, much larger than when a random set of expert annotations are added"], "all_classes": null, "length": 4399}
{"pred": "Unanswerable", "answers": ["Women represent 33.16% of the speakers"], "all_classes": null, "length": 4055}
{"pred": "Yes. According to the article, the approach achieves state of the art results on the English-German dataset.", "answers": ["the English-German dataset"], "all_classes": null, "length": 1833}
{"pred": "Yes, the paper compares the proposed model to several strong baseline models, including BIBREF18, BIBREF20, BIBREF24, BIBREF26, BIBREF27, BIBREF30, BIBREF33, BIBREF34, BIBREF35, BIBREF36, BIBREF37, BIBREF40, BIBREF41, BIBREF42, BIBREF43, BIBREF444, BIBREF45, BIBREF46, BIB", "answers": ["Baseline models are:\n- Chen et al., 2015a\n- Chen et al., 2015b\n- Liu et al., 2016\n- Cai and Zhao, 2016\n- Cai et al., 2017\n- Zhou et al., 2017\n- Ma et al., 2018\n- Wang et al., 2019"], "all_classes": null, "length": 3629}
{"pred": "Unanswerable", "answers": ["probabilistic model", "Logistic Regression, Multilayer Perceptron"], "all_classes": null, "length": 4475}
{"pred": "BIBREF17, BIBREF18, BIBREF19, BIBREF24, BIBREF25, BIBREF26, BIBREF18\n\n", "answers": ["BIBREF17, BIBREF18, TensiStrength BIBREF13, TwitterNLP BIBREF6, BIBREF19, CogComp-NLP BIBREF20, Stanford NLP NER BIBREF21", "BIBREF23, BIBREF17, BIBREF18, BIBREF19, BIBREF24, BIBREF25, BIBREF26"], "all_classes": null, "length": 1452}
{"pred": "Experiments are performed on the SQuAD dataset.", "answers": ["SQuAD", "SQuAD"], "all_classes": null, "length": 3757}
{"pred": "Unanswerable. The article does not discuss or compare any existing approaches for learning geographic location embeddings using Flickr tags and structured data.", "answers": ["BOW-Tags, BOW-KL(Tags), BOW-All, GloVe"], "all_classes": null, "length": 4658}
{"pred": "Yes", "answers": ["Yes", "Yes"], "all_classes": null, "length": 1687}
{"pred": " They used the following datasets for evaluation:\n\n- CSAT dataset for CSAT prediction consisting of spoken transcripts (automatic via ASR)\n- 20 newsgroups dataset for topic identification task consisting of written text\n- Fisher Phase 1 corpus for topic identification task consisting of spoken transcripts (manual)\n\nSo the answer is: \n\nYes", "answers": ["CSAT dataset, 20 newsgroups, Fisher Phase 1 corpus", "CSAT dataset , 20 newsgroups, Fisher Phase 1 corpus"], "all_classes": null, "length": 2652}
{"pred": "Yes, the IMDb movie review dataset is used for sentiment classification experiments.", "answers": ["the IMDb movie review dataset BIBREF17", "IMDb movie review"], "all_classes": null, "length": 3432}
{"pred": "No", "answers": ["Yes", "Yes"], "all_classes": null, "length": 1464}
{"pred": "No", "answers": ["No"], "all_classes": null, "length": 1441}
{"pred": "Invertible transformations have been explored before in independent components analysis, gaussianization, and deep density models. Here, we generalize this style of approach to structured learning, and augment it with discrete latent variables. Under the invertibility condition, we derive a learning algorithm and give another view of our approach revealed by the objective function.", "answers": ["The neural projector must be invertible.", "we constrain our neural projector with two requirements: (1) INLINEFORM0 and (2) INLINEFORM1 exists"], "all_classes": null, "length": 4323}
{"pred": "The proposed qualitative annotation schema looks like:\n\n1. Problem definition: Define the task of machine reading comprehension and the target application of the proposed methodology.\n\n2. Dimensions of interest: Categorize the gold standards according to linguistic complexity, required reasoning, factual correctness, and knowledge.\n\n3. Annotation task: Annotate question, answer and passage triples with the proposed categories.\n\n4. Qualitative analysis: Analyze the annotated samples to identify issues with factual correctness, lexical cues and semantics-altering grammatical modifiers.", "answers": ["The resulting taxonomy of the framework is shown in Figure FIGREF10", "FIGREF10"], "all_classes": null, "length": 4958}
{"pred": "The WikiSmall dataset has 89,042 sentence pairs for training and 100 pairs for testing. The WikiLarge dataset has 296,402 sentence pairs for training and 8 (reference) simplifications for 2,359 sentences split into 2,000 for development and 359 for testing.\n\n", "answers": ["training set has 89,042 sentence pairs, and the test set has 100 pairs, training set contains 296,402, 2,000 for development and 359 for testing", "WikiSmall  89 142 sentence pair and  WikiLarge 298 761 sentence pairs. "], "all_classes": null, "length": 2266}
{"pred": "Yes. The paper mentions several baselines in the experiments section, including vanilla ST, pre-training baselines, multi-task baselines, and triangle+pre-train baseline.", "answers": ["Vanilla ST baseline, encoder pre-training, in which the ST encoder is initialized from an ASR model, decoder pre-training, in which the ST decoder is initialized from an MT model, encoder-decoder pre-training, where both the encoder and decoder are pre-trained, many-to-many multi-task model where the encoders and decoders are derived from pre-trained ASR and MT models, Triangle+pre-train: BIBREF18 DBLP:conf/naacl/AnastasopoulosC18 proposed a triangle multi-task strategy for speech translation", "Vanilla ST baseline, Pre-training baselines, Multi-task baselines, Many-to-many+pre-training, Triangle+pre-train", "Vanilla ST baseline: The vanilla ST BIBREF9 has only a speech encoder and a decoder. It is trained from scratch on the ST-TED corpus.\n\nPre-training baselines: We conduct three pre-training baseline experiments: 1) encoder pre-training, in which the ST encoder is initialized from an ASR model; 2) decoder pre-training, in which the ST decoder is initialized from an MT model; and 3) encoder-decoder pre-training, where both the encoder and decoder are pre-trained. The ASR model has the same architecture with vanilla ST model, trained on the mixture of ST-TED and TED-LIUM2 corpus. The MT model has a text encoder and decoder with the same architecture of which in TCEN. It is first trained on WMT data (out-of-domain) and then fine-tuned on in-domain data.\n\nMulti-task baselines: We also conduct three multi-task baseline experiments including one-to-many setting, many-to-one setting, and many-to-many setting. In the first two settings, we train the model with $\\alpha _{st}=0.75$ while $\\alpha _{asr}=0.25$ or $\\alpha _{mt}=0.25$. For many-to-many setting, we use $\\alpha _{st}=0.6, \\alpha _{asr}=0.2$ and $\\alpha _{mt}=0.2$.. For MT task, we use only in-domain data.\n\nMany-to-many+pre-training: We train a many-to-many multi-task model where the encoders and decoders are derived from pre-trained ASR and MT models. "], "all_classes": null, "length": 4704}
{"pred": "English", "answers": ["Unanswerable", "English"], "all_classes": null, "length": 4513}
{"pred": "Yes", "answers": ["linear SVM, bidirectional Long Short-Term-Memory (BiLSTM), Convolutional Neural Network (CNN)", "linear SVM, bidirectional Long Short-Term-Memory (BiLSTM), Convolutional Neural Network (CNN)", "linear SVM trained on word unigrams,  bidirectional Long Short-Term-Memory (BiLSTM),  Convolutional Neural Network (CNN) "], "all_classes": null, "length": 2250}
{"pred": "No", "answers": ["No"], "all_classes": null, "length": 1561}
{"pred": "Unanswerable", "answers": ["Pretrained word embeddings  were not used", "GloVe, Edinburgh embeddings BIBREF14, Emoji embeddings BIBREF16"], "all_classes": null, "length": 1771}
{"pred": "The results on the new dataset were presented in the \"Experiments and Results\" section. They showed that their personalized models outperformed the baseline models in terms of perplexity, BLEU-1/4, ROUGE-L, recipe-level coherence, and step entailment.", "answers": ["average recipe-level coherence scores of 1.78-1.82, human evaluators preferred personalized model outputs to baseline 63% of the time"], "all_classes": null, "length": 2666}
{"pred": "The combination of rewards for reinforcement learning is:\n\n- irony reward: the difference between the irony score of the input sentence and that of the output sentence\n- sentiment reward: one minus the absolute value of the difference between the standardized sentiment score of the input sentence and that of the generated sentence\n\nThis is according to the article.", "answers": ["irony accuracy, sentiment preservation", " irony accuracy and sentiment preservation"], "all_classes": null, "length": 4592}
{"pred": " The authors demonstrate several limitations of their model:\n\n1. The generated English poems may not work well with Shakespeare style transfer for paintings that have a low average content score. This happens when the style transfer dataset does not have similar words in the training set of sentences.\n\n2. The model may not be able to capture more complex paintings that have multiple objects or scenes.\n\n3. The model may not be able to generate poems that are very creative or have a high style score.\n\n4. The model requires an intermediate representation to combine the image-to-poem and text-", "answers": ["Since we do not have an end-to-end dataset, the generated English poem may not work well with Shakespeare style transfer", "we do not have an end-to-end dataset, the generated English poem may not work well with Shakespeare style transfer as shown in Figure FIGREF12 for \"Starry Night\" with a low average content score"], "all_classes": null, "length": 1651}
{"pred": "Yes", "answers": ["Affective Text, Fairy Tales, ISEAR", " Affective Text dataset, Fairy Tales dataset, ISEAR dataset"], "all_classes": null, "length": 3390}
{"pred": "The distribution results are shown in Tables TABREF23 and Figures FIGREF24-FIGREF35.", "answers": ["Distributions of Followers, Friends and URLs are significantly different between the set of tweets containing fake news and those non containing them, but for Favourites, Mentions, Media, Retweets and Hashtags they are not significantly different"], "all_classes": null, "length": 3164}
{"pred": "The dataset of hashtags is created by crowdsourcing from the Stanford Sentiment Analysis dataset. The authors first extract hashtags from the tweets in the dataset and then manually segment them into word sequences. They also use an additional dataset of 1,108 hashtags to train and evaluate their model.", "answers": ["1,268 randomly selected tweets in the Stanford Sentiment Analysis Dataset BIBREF36, all 12,594 unique English hashtags and their associated tweets from the same Stanford dataset", "Stanford Sentiment Analysis Dataset BIBREF36"], "all_classes": null, "length": 3756}
{"pred": "Unanswerable", "answers": ["Unanswerable", "Unanswerable"], "all_classes": null, "length": 3794}
{"pred": " Word subspace can represent most of the variability of the data in each class of the Reuters-8 database.", "answers": ["Word vectors, usually in the context of others within the same class"], "all_classes": null, "length": 5151}
{"pred": "The article does not mention any specific baseline model used. Therefore, the answer is \"unanswerable\".", "answers": ["For Article-Entity placement, they consider two baselines: the first one using only salience-based features, and the second baseline checks if the entity appears in the title of the article. \n\nFor Article-Section Placement, they consider two baselines: the first picks the section with the highest lexical similarity to the article, and the second one picks the most frequent section.", "B1. The first baseline uses only the salience-based features by Dunietz and Gillick BIBREF11 ., B2. The second baseline assigns the value relevant to a pair INLINEFORM0 , if and only if INLINEFORM1 appears in the title of INLINEFORM2 .\n\n, S1: Pick the section from template INLINEFORM0 with the highest lexical similarity to INLINEFORM1 : S1 INLINEFORM2, S2: Place the news into the most frequent section in INLINEFORM0"], "all_classes": null, "length": 7891}
{"pred": "No", "answers": ["Yes", "Unanswerable"], "all_classes": null, "length": 2000}
{"pred": "Unanswerable", "answers": ["Unanswerable", "Unanswerable"], "all_classes": null, "length": 2410}
{"pred": "The paper does not mention any specific dataset. The question is unanswerable based on the information provided in the article.", "answers": [" high-quality datasets  from SemEval-2016 “Sentiment Analysis in Twitter” task", " SemEval-2016 “Sentiment Analysis in Twitter”"], "all_classes": null, "length": 2738}
{"pred": "Small BERT", "answers": ["small BERT", "small BERT"], "all_classes": null, "length": 1999}
{"pred": "Unanswerable", "answers": ["No", "No"], "all_classes": null, "length": 6391}
{"pred": "No", "answers": ["Yes", "Yes"], "all_classes": null, "length": 3472}
{"pred": "Yes, they reported their performance on emotion detection in Table TABREF26.", "answers": ["Answer with content missing: (Table 3) Best author's model B-M average micro f-score is 0.409, 0.459, 0.411 on Affective, Fairy Tales and ISEAR datasets respectively. "], "all_classes": null, "length": 3410}
{"pred": "The INLINEFORM0 tagging scheme is employed.", "answers": ["A new tagging scheme that tags the words before and after the pun as well as the pun words.", "a new tagging scheme consisting of three tags, namely { INLINEFORM0 }"], "all_classes": null, "length": 2974}
{"pred": "No", "answers": ["No", "No"], "all_classes": null, "length": 2413}
{"pred": "Unanswerable\n\nThe article does not define or discuss how they define or measure the robustness of a model. The focus is on leveraging prior knowledge to improve the model's performance, but does not provide any metrics or evaluation of the model's robustness.", "answers": ["ability to accurately classify texts even when the amount of prior knowledge for different classes is unbalanced, and when the class distribution of the dataset is unbalanced", "Low sensitivity to bias in prior knowledge"], "all_classes": null, "length": 3609}
{"pred": "SBERT is compared to InferSent, Universal Sentence Encoder, average GloVe embeddings, and Polyencoders.", "answers": ["GloVe, BERT, Universal Sentence Encoder, TF-IDF, InferSent", "Avg. GloVe embeddings, Avg. fast-text embeddings, Avg. BERT embeddings, BERT CLS-vector, InferSent - GloVe and Universal Sentence Encoder."], "all_classes": null, "length": 3862}
{"pred": " English: +0.29\nChinese: +0.97, +2.36\n\nUnanswerable: The article does not provide F1 score improvements for NER task for English and Chinese datasets.", "answers": ["English datasets including CoNLL2003 and OntoNotes5.0, our proposed method outperforms BERT-MRCBIBREF38 by +0.29 and +0.96 respectively, Chinese datasets, achieving F1 improvements by +0.97 and +2.36 on MSRA and OntoNotes4.0, respectively", "For English datasets including CoNLL2003 and OntoNotes5.0, our proposed method outperforms BERT-MRCBIBREF38 by +0.29 and +0.96 respectively., huge performance boosts on Chinese datasets, achieving F1 improvements by +0.97 and +2.36 on MSRA and OntoNotes4.0, respectively"], "all_classes": null, "length": 3586}
{"pred": "The article tests their conflict method on two tasks: Quora duplicate question pair detection and ranking questions in Bing's People Also Ask.", "answers": ["Task 1: Quora Duplicate Question Pair Detection, Task 2: Ranking questions", "Quora Duplicate Question Pair Detection, Ranking questions in Bing's People Also Ask"], "all_classes": null, "length": 2577}
{"pred": "Unanswerable\n\nThe article does not mention any specific baselines or comparisons.", "answers": ["Various tree structured neural networks including variants of Tree-LSTM, Tree-based CNN, RNTN, and non-tree models including variants of LSTMs, CNNs, residual, and self-attention based networks", "Sentence classification baselines: RNTN (Socher et al. 2013), AdaMC-RNTN (Dong et al. 2014), TE-RNTN (Qian et al. 2015), TBCNN (Mou et al. 2015), Tree-LSTM (Tai, Socher, and Manning 2015), AdaHT-LSTM-CM (Liu, Qiu, and Huang 2017), DC-TreeLSTM (Liu, Qiu, and Huang 2017), TE-LSTM (Huang, Qian, and Zhu 2017), BiConTree (Teng and Zhang 2017), Gumbel Tree-LSTM (Choi, Yoo, and Lee 2018), TreeNet (Cheng et al. 2018), CNN (Kim 2014), AdaSent (Zhao, Lu, and Poupart 2015), LSTM-CNN (Zhou et al. 2016), byte-mLSTM (Radford, Jozefowicz, and Sutskever 2017), BCN + Char + CoVe (McCann et al. 2017), BCN + Char + ELMo (Peters et al. 2018). \nStanford Natural Language Inference baselines: Latent Syntax Tree-LSTM (Yogatama et al. 2017), Tree-based CNN (Mou et al. 2016), Gumbel Tree-LSTM (Choi, Yoo, and Lee 2018), NSE (Munkhdalai and Yu 2017), Reinforced Self- Attention Network (Shen et al. 2018), Residual stacked encoders: (Nie and Bansal 2017), BiLSTM with generalized pooling (Chen, Ling, and Zhu 2018)."], "all_classes": null, "length": 4781}
{"pred": "Hierarchical Residual BiLSTM (HR-BiLSTM)", "answers": ["answer questions by obtaining information from KB tuples ", "hierarchical matching between questions and relations with residual learning"], "all_classes": null, "length": 4527}
{"pred": "Yes, the article mentions two baseline models:\n\n- A simple Encoder-Decoder model with ingredient attention (Enc-Dec)\n- A Nearest-Neighbor model (NN) adapted from the Neural Checklist Model of BIBREF0", "answers": ["name-based Nearest-Neighbor model (NN), Encoder-Decoder baseline with ingredient attention (Enc-Dec)"], "all_classes": null, "length": 2655}
{"pred": "Unanswerable", "answers": ["spot patterns by just looking at a collection of images, tag all descriptions with part-of-speech information, I applied Louvain clustering", "Looking for adjectives marking the noun \"baby\" and also looking for most-common adjectives related to certain nouns using POS-tagging"], "all_classes": null, "length": 2204}
{"pred": "French", "answers": ["English, French, German ", "French, English, Spanish, Italian, Portuguese, Hebrew, Arabic"], "all_classes": null, "length": 2285}
{"pred": "The authors experimented with stacked LSTMs, cell-aware stacked LSTMs, and bidirectional cell-aware stacked LSTMs.", "answers": ["Stacked LSTMs, Cell-aware Stacked LSTMs, Sentence Encoders, Top-layer Classifiers"], "all_classes": null, "length": 3210}
{"pred": "No", "answers": ["Yes", "Unanswerable"], "all_classes": null, "length": 6169}
{"pred": "ILP-based summarization.", "answers": ["LSA, TextRank, LexRank and ILP-based summary.", "LSA, TextRank, LexRank"], "all_classes": null, "length": 3045}
{"pred": "The previous state of the art for this task was proposed by BIBREF0 .", "answers": ["hLSTM", "hLSTM"], "all_classes": null, "length": 3725}
{"pred": "Unanswerable", "answers": ["Based on table results provided changing directed to undirected edges had least impact - max abs difference of 0.33 points on all three datasets."], "all_classes": null, "length": 4180}
{"pred": "German corpora DTA18 and DTA19.", "answers": ["DTA18, DTA19", "Diachronic Usage Relatedness (DURel) gold standard data set"], "all_classes": null, "length": 1908}
{"pred": "Yes", "answers": ["Hindi, English, Kannada, Telugu, Assamese, Bengali and Malayalam", "Kannada, Hindi, Telugu, Malayalam, Bengali,  English and Assamese (in table, missing in text)"], "all_classes": null, "length": 2453}
{"pred": "Unanswerable\n\nThe article does not provide any information on the model performance on target language reading comprehension.", "answers": ["Table TABREF6, Table TABREF8", "when testing on English, the F1 score of the model training on Chinese (Zh) is 53.8,  F1 score is only 44.1 for the model training on Zh-En"], "all_classes": null, "length": 2492}
{"pred": "The proposed model ALOHA achieves a significant improvement over the baselines, with an average Hits@1/20 score of 40.67% compared to the baseline score of 24.34%. This represents a 67% improvement in performance.\n\n", "answers": ["Metric difference between Aloha and best baseline score:\nHits@1/20: +0.061 (0.3642 vs 0.3032)\nMRR: +0.0572(0.5114 vs 0.4542)\nF1: -0.0484 (0.3901 vs 0.4385)\nBLEU: +0.0474 (0.2867 vs 0.2393)"], "all_classes": null, "length": 5151}
{"pred": "Unanswerable\n\nThe article does not provide any quantitative comparison of the improvement gained from ARAMLE compared to other baselines.", "answers": ["ARAM has achieved improvement over all baseline methods using reverese perplexity and  slef-BLEU metric.  The maximum reverse perplexity improvement 936,16 is gained for EMNLP2017  WMT  dataset and  48,44 for COCO dataset.", "Compared to the baselines, ARAML does not do better in terms of perplexity on COCO and EMNLP 2017 WMT datasets, but it does by up to 0.27 Self-BLEU points on COCO and 0.35 Self-BLEU on EMNLP 2017 WMT. In terms of Grammaticality and Relevance, it scores better than the baselines on up to 75.5% and 73% of the cases respectively."], "all_classes": null, "length": 3796}
{"pred": " The authors present evidence that the model can capture some biases in data annotation and collection by examining the results of misclassifications. They find that some misclassifications are due to biases in the data collection and annotation rules, not the model itself. For example, they find that some tweets containing African American Vernacular English and geographic restrictions are oversampled, leading to misclassifications. They also manually inspect some samples and find that the model can correctly classify tweets containing implicit hatred content, but misclassify hate content as offensive.", "answers": ["The authors showed few tweets where neither and implicit hatred content exist but the model was able to discriminate"], "all_classes": null, "length": 4119}
{"pred": "No. Only the neural baselines were tested to compare with the neural baseline.", "answers": ["SVM, No-Answer Baseline (NA) , Word Count Baseline, Human Performance", "No-Answer Baseline (NA), Word Count Baseline, Human Performance"], "all_classes": null, "length": 3855}
{"pred": "The size of the dataset is not mentioned in the article. Therefore, the answer is \"unanswerable\".", "answers": ["Dataset contains 3606 total sentences and 79087 total entities.", "ILPRL contains 548 sentences, OurNepali contains 3606 sentences"], "all_classes": null, "length": 2843}
{"pred": "No. The article does not discuss F1 score improvements for paraphrase identification tasks.", "answers": ["Using DSC loss improves the F1 score by +0.58 for MRPC and +0.73 for QQP", "+0.58"], "all_classes": null, "length": 3566}
{"pred": "Unanswerable", "answers": ["Answer with content missing: (Whole Method and Results sections) The primary dataset we use is the ERP data collected and computed by Frank et al. (2015), and we also use behavioral data (eye-tracking data and self-paced reading times) from Frank et al. (2013) which were collected on the same set of 205 sentences.\nSelect:\n- ERP data collected and computed by Frank et al. (2015)\n- behavioral data (eye-tracking data and self-paced reading times) from Frank et al. (2013)", "the ERP data: BIBREF0"], "all_classes": null, "length": 1971}
{"pred": "Unanswerable", "answers": ["7 phonemic/syllabic ( /iy/, /piy/, /tiy/, /diy/, /uw/, /m/, /n/ ) as well as 4 words(pat, pot, knew and gnaw)", "KARA ONE BIBREF17 , composed of multimodal data for stimulus-based, imagined and articulated speech state corresponding to 7 phonemic/syllabic ( /iy/, /piy/, /tiy/, /diy/, /uw/, /m/, /n/ ) as well as 4 words(pat, pot, knew and gnaw)"], "all_classes": null, "length": 2379}
{"pred": "The article mentions the following baselines are used for evaluation:\n\n- Pointer-Gen\n-RL+ROUGE\n- Pointer-Gen-RL-SEN\n- Pointer-Gen-ARL-SEN\n\nSo the answer is \"yes\".", "answers": ["Pointer-Gen, Pointer-Gen+Pos, Pointer-Gen+Same-FT, Pointer-Gen+Pos-FT, Pointer-Gen+RL-ROUGE, Pointer-Gen+RL-SEN"], "all_classes": null, "length": 4085}
{"pred": "The article mentions the following learning models are used on the dataset:\n\n1. Naive Bayes\n2. Logistic Regression\n3. Support Vector Machine\n4. Random Forests\n5. Gradient Boosted Trees\n6. Convolutional Neural Networks\n7. Recurrent Neural Networks\n\nSo the answer is: All of the above.", "answers": ["Naïve Bayes (NB), Logistic Regression (LR), Support Vector Machine (SVM), Random Forests (RF), Gradient Boosted Trees (GBT),  Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN)", "Naïve Bayes (NB), Logistic Regression (LR), Support Vector Machine (SVM), Random Forests (RF), Gradient Boosted Trees (GBT), CNN, RNN"], "all_classes": null, "length": 2074}
{"pred": "The article mentions two types of language model architectures:\n\n1. A bi-directional language model to augment the sequence to sequence encoder\n2. A uni-directional model to augment the decoder\n\nSo the answer is: \n\nBoth bi-directional and uni-directional language model architectures are used.", "answers": ["uni-directional model to augment the decoder", "bi-directional language model to augment the sequence to sequence encoder ,  uni-directional model to augment the decoder"], "all_classes": null, "length": 1914}
{"pred": "Dynamic weight adjusting strategy associates each training example with a weight in proportion to $(1-p)$, where $p$ is the probability of the example being correctly classified. The weight dynamically changes as training proceeds.", "answers": ["One can think $(1-p_{i1})$ as a weight associated with each example, which changes as training proceeds. The intuition of changing $p_{i1}$ to $(1-p_{i1}) p_{i1}$ is to push down the weight of easy examples. For easy examples whose probability are approaching 0 or 1, $(1-p_{i1}) p_{i1}$ makes the model attach significantly less focus to them. Figure FIGREF23 gives gives an explanation from the perspective in derivative: the derivative of $\\frac{(1-p)p}{1+(1-p)p}$ with respect to $p$ approaches 0 immediately after $p$ approaches 0, which means the model attends less to examples once they are correctly classified.", "associates each training example with a weight in proportion to $(1-p)$, and this weight dynamically changes as training proceeds"], "all_classes": null, "length": 3640}
{"pred": "Yes. The article describes the results of the proposed strategies for exploring text-adventure games, including the KG-A2C-chained and KG-A2C-Explore methods. The results show that the knowledge graph-based methods outperform the baseline methods in passing bottlenecks and reaching higher scores.", "answers": ["Reward of 11.8 for the A2C-chained model, 41.8 for the KG-A2C-chained model, 40 for A2C-Explore and 44 for KG-A2C-Explore.", "KG-A2C-chained and KG-A2C-Explore both pass the bottleneck of a score of 40"], "all_classes": null, "length": 2443}
{"pred": " An individual model consists of monolingual Bayesian models for each language and crosslingual latent variables to incorporate soft role agreement between aligned constituents.", "answers": ["Bayesian model of garg2012unsupervised as our base monolingual model"], "all_classes": null, "length": 3701}
{"pred": "Unanswerable", "answers": ["Unanswerable", "Original transcription was labeled with additional labels in [] brackets with nonstandard pronunciation."], "all_classes": null, "length": 3018}
{"pred": "Semichar architecture refers to a type of word recognition model that processes a sentence of words with misspelled characters, predicting the correct words at each step.", "answers": ["A semi-character based RNN (ScRNN) treats the first and last characters individually, and is agnostic to the ordering of the internal characters", "processes a sentence of words with misspelled characters, predicting the correct words at each step"], "all_classes": null, "length": 4186}
{"pred": "Bulgarian, Croatian, Czech, Danish, English, German, Indonesian, Italian, Norwegian, Persian, Polish, Portuguese, Slovenian, Spanish, Swedish.\n\n", "answers": ["Bulgarian, Croatian, Czech, Danish, English, French, German, Indonesian, Italian, Norwegian, Persian, Polish, Portuguese, Slovenian, Spanish and Swedish", "Bulgarian, Croatian, Czech, Danish, English, French, German, Indonesian, Italian, Norwegian, Persian, Polish, Portuguese, Slovenian, Spanish , Swedish"], "all_classes": null, "length": 2697}
{"pred": "Yes", "answers": ["NCEL consistently outperforms various baselines with a favorable generalization ability"], "all_classes": null, "length": 4113}
{"pred": " Yes", "answers": ["Yes", "Yes"], "all_classes": null, "length": 4592}
{"pred": "Yes", "answers": ["error detection system by Rei2016", "error detection system by Rei2016"], "all_classes": null, "length": 2132}
{"pred": "The article does not mention where they obtained the annotated clinical notes from. Therefore, the answer is \"unanswerable\".", "answers": ["clinical notes from the CE task in 2010 i2b2/VA", "clinical notes from the CE task in 2010 i2b2/VA "], "all_classes": null, "length": 3432}
{"pred": "MSMSMSMS.MS.MS.MS.MS.MS.MS.MS.\nMS.MS.MS.MS.\nMS.MS.MS.MS.MS.\nMS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.MS.", "answers": ["ecause this process is similar to the cloze task in BERT's pre-train process, therefore by using the ability of the contextual language model the decoder can generate more fluent and natural sequences."], "all_classes": null, "length": 3919}
{"pred": "Unanswerable", "answers": ["Unanswerable", " Paraphrase Database (PPDB) ,  book corpus", "Unanswerable"], "all_classes": null, "length": 1902}
{"pred": "TF-IDF features and keywords extracted using TF-IDF and LDA.", "answers": ["Unanswerable"], "all_classes": null, "length": 2108}
{"pred": "The dataset is annotated with 9,473 annotations for 9,30 tweets. Each tweet is annotated as no evidence of depression or evidence of depression, and if evidence of depression, it is further annotated with one or more depressive symptoms.", "answers": ["no evidence of depression, depressed mood, disturbed sleep, fatigue or loss of energy", "The annotations are based on evidence of depression and further annotated by the depressive symptom if there is evidence of depression"], "all_classes": null, "length": 1947}
{"pred": "Yes, they evaluated on eight biomedical NER tasks.", "answers": ["BC5CDR-disease, NCBI-disease, BC5CDR-chem, BC4CHEMD, BC2GM, JNLPBA, LINNAEUS, Species-800", "BC5CDR-disease, NCBI-disease, BC5CDR-chem, BC4CHEMD, BC2GM, JNLPBA, LINNAEUS, Species-800"], "all_classes": null, "length": 2800}
{"pred": "Unanswerable", "answers": ["using the machine translation platform Apertium ", "machine translation platform Apertium BIBREF5"], "all_classes": null, "length": 2423}
{"pred": "The article does not mention any specific model they used for their system. Therefore, the answer is \"unanswerable\".", "answers": ["AllWords model by counting the frequencies of all the remaining words and training a multinomial Naive Bayes classifier"], "all_classes": null, "length": 4177}
{"pred": "The baseline for this task was a simple logistic regression classifier that represented the input instances with a single feature: the length of the sentence.", "answers": ["The baseline system for the SLC task is a very simple logistic regression classifier with default parameters. The baseline for the FLC task generates spans and selects one of the 18 techniques randomly.", "SLC task is a very simple logistic regression classifier, FLC task generates spans and selects one of the 18 techniques randomly"], "all_classes": null, "length": 3001}
{"pred": "Yes", "answers": ["They compare with the following models: by Pedersen (2017), by Pramanick and Das (2017), by Mikhalkova and Karyakin (2017),  by Vadehra (2017), Indurthi and Oota (2017), by Vechtomova (2017), by (Cai et al., 2018), and CRF."], "all_classes": null, "length": 2991}
{"pred": "Unanswerable", "answers": ["By assigning a political bias label to each news article and training only on left-biased or right-biased outlets of both disinformation and mainstream domains", "we also account for political biases inherent to different news sources, referring to the procedure proposed in BIBREF2 to label different outlets. Overall we show that we are able to classify credible vs non-credible diffusion networks (and consequently news articles) with high accuracy (AUROC up to 94%), even when accounting for the political bias of sources (and training only on left-biased or right-biased articles). We observe that the layer of mentions alone conveys useful information for the classification, denoting a different usage of this functionality when sharing news belonging to the two news domains. We also show that most discriminative features, which are relative to the breadth and depth of largest cascades in different layers, are the same across the two countries."], "all_classes": null, "length": 4882}
{"pred": "The ancient Chinese dataset comes from 1.7K bilingual ancient-modern Chinese articles from the internet.", "answers": ["ancient Chinese history records in several dynasties (about 1000BC-200BC) and articles written by celebrities of that era", "Ancient Chinese history records in several dynasties and articles written by celebrities during 1000BC-200BC collected from the internet "], "all_classes": null, "length": 3722}
{"pred": "Unanswerable", "answers": ["English", "English ", "English"], "all_classes": null, "length": 2240}
{"pred": "Unanswerable", "answers": ["Answer with content missing: (Data section) Chinese with version 5.1 of the Chinese Penn Treebank (CTB)"], "all_classes": null, "length": 2545}
{"pred": " The UTCNN model has 3 convolutional layers and 1 maximum pooling layer.", "answers": ["eight layers"], "all_classes": null, "length": 4487}
{"pred": "The paper does not mention any specific dataset used. The paper only mentions \"existing work\" and \"structured datasets\" without providing any specific details. Therefore, the answer is \"unanswerable\".", "answers": [" the same datasets as BIBREF7", "same datasets as BIBREF7"], "all_classes": null, "length": 4661}
{"pred": "NUBes-PHI and MEDDOCAN", "answers": ["MEDDOCAN, NUBes-PHI", "MEDDOCAN, NUBes "], "all_classes": null, "length": 4528}
{"pred": "No. The article does not mention any traditional linguistics features used. The features used were derived from eye-movement data.", "answers": ["Unanswerable"], "all_classes": null, "length": 3543}
{"pred": "The article does not provide any specific metrics to establish that LiLi makes chatbots more knowledgeable and better at learning and conversation. The article only discusses the predictive performance and strategy formulation ability of LiLi as evaluated through various metrics like MCC and F1 score.\n\nTherefore, the answer is: Unanswerable.", "answers": ["Coverage, Avg. MCC and avg. +ve F1 score", "strategy formulation ability, we introduce a measure called Coverage( INLINEFORM0 ), To evaluate the predictive performance, we use Avg. MCC and avg. +ve F1 score"], "all_classes": null, "length": 5869}
{"pred": "No", "answers": ["Yes", "No"], "all_classes": null, "length": 1910}
{"pred": "The article mentions that the targets of the stance detection are two popular sports clubs in Turkey, specifically Galatasaray and Fenerbahçe.", "answers": ["Galatasaray, Fenerbahçe", "Galatasaray , Fenerbahçe "], "all_classes": null, "length": 2234}
{"pred": "The paper conducts experiments on the following:\n\n1. Automatic evaluation of the models on the transformation from non-ironic sentences to ironic sentences.\n\n2. Human evaluation of the generated sentences using a ranking system.\n\n3. Additional experiments on the transformation from ironic sentences to non-ironic sentences.\n\n4. Analysis of the errors and limitations of the model.\n\nSo the answer is: Experiments are conducted.", "answers": ["Irony Classifier, Sentiment Classifier for Irony, Sentiment Classifier for Non-irony, transformation from ironic sentences to non-ironic sentences"], "all_classes": null, "length": 4600}
{"pred": "Yes, the paper proposes Gaussian-masked directional multi-head attention as a variant of self-attention to capture localness and directional information. The Gaussian weight matrix is used to adjust the attention weights based on the distance between characters. The Hadamard product of the Gaussian weight matrix and the score matrix is used to generate the attention weights.", "answers": ["pays attentions to adjacent characters and casts a localness relationship between the characters as a fixed Gaussian weight assuming the weight relies on the distance between characters", "Gaussian-masked directional attention can be described as a function to map queries and key-value pairs to the representation of input, Gaussian-masked directional attention expects to pay attention to the adjacent characters of each positions and cast the localness relationship between characters as a fix Gaussian weight for attention, Gaussian weight only relys on the distance between characters"], "all_classes": null, "length": 3612}
{"pred": "Unanswerable\n\nThe article only mentions Facebook status updates as the dataset they used for causal explanation analysis. They do not consider any other social media platforms.", "answers": ["Facebook status update messages", "Facebook status update messages"], "all_classes": null, "length": 4005}
{"pred": " Baseline features are the features extracted directly from the CNN network without using any pre-trained models.", "answers": [" The features extracted from CNN."], "all_classes": null, "length": 4855}
{"pred": "The article does not vary any hyperparameters in the experiments on the four tasks. The number of clusters and the dimensionality of the word embeddings were fixed for each task.", "answers": ["number of clusters, seed value in clustering, selection of word vectors, window size and dimension of embedding", "different number of clusters, different embeddings"], "all_classes": null, "length": 2753}
{"pred": "Yes", "answers": ["column Ens Test in Table TABREF19"], "all_classes": null, "length": 2424}
{"pred": "The corpus consists of 53 documents, which contain an average number of 156.1 sentences per document, each with 19.5 tokens on average. The corpus comprises 8,275 sentences and 167,739 words in total.", "answers": ["8,275 sentences and 167,739 words in total", "The corpus comprises 8,275 sentences and 167,739 words in total."], "all_classes": null, "length": 2669}
{"pred": "No", "answers": ["Unanswerable", "Unanswerable"], "all_classes": null, "length": 2764}
{"pred": "Yes. They consider text categorization as an example NLP task.", "answers": ["text classification for themes including sentiment, web-page, science, medical and healthcare"], "all_classes": null, "length": 3591}
{"pred": "The BERT-QC model is compared to the following previous methods:\n\n- CNN model\n- BERT model\n- rule-based methods\n- customized rule-based pattern matching\n- learned methods like CPT2\n\nSo the answer is \"yes\".", "answers": ["bag-of-words model, CNN"], "all_classes": null, "length": 5838}
{"pred": "The article does not provide information on the size of the training sets for the different versions of ELMo. Therefore, this question is unanswerable based on the information provided.", "answers": ["By 14 times.", "up to 1.95 times larger"], "all_classes": null, "length": 3294}
{"pred": "The dataset contains 64,000 sentences.", "answers": ["3606", "6946"], "all_classes": null, "length": 2835}
{"pred": " They compare to Eusboost and MWMOTE.", "answers": ["MLP", "Eusboost, MWMOTE"], "all_classes": null, "length": 2474}
{"pred": "Yes", "answers": ["Yes", "Yes"], "all_classes": null, "length": 3784}
{"pred": "No", "answers": ["Yes", "Unanswerable"], "all_classes": null, "length": 4320}
{"pred": " Yes, their highest MRR score was 0.6103.", "answers": ["0.5115", "0.6103"], "all_classes": null, "length": 6810}
{"pred": "The article evaluates on the Wall Street Journal (WSJ) portion of the Penn Treebank.", "answers": [" Wall Street Journal (WSJ) portion of the Penn Treebank", "Unanswerable"], "all_classes": null, "length": 4327}
{"pred": "The authors provide evidence by stating that \"building models under these frameworks requires a large overhead of mastering these framework details.\"", "answers": ["By conducting a survey among engineers", "Unanswerable"], "all_classes": null, "length": 1692}
{"pred": "The article states that they achieve state-of-the-art results on the SimpleQuestions and WebQSP benchmarks.\n\n\"Our proposed HR-BiLSTM outperformed the best baselines on both tasks by margins of 2-3% (p $<$ 0.001 and 0.01 compared to the best baseline BiLSTM w/ words on SQ and WQ respectively).\"\n\nSo the answer is:\n\nYes, they achieve state-of-the-art results on the SimpleQuestions and WebQSP benchmark", "answers": ["SimpleQuestions, WebQSP", "WebQSP, SimpleQuestions"], "all_classes": null, "length": 4524}
