# Datasets and sources of raw data

## Raw / unclean datasets

- [Boston Housing Dataset (archive contains unclean dataset)](https://github.com/neomatrix369/awesome-ai-ml-dl/releases/tag/v0.1) | [Download](https://github.com/neomatrix369/awesome-ai-ml-dl/releases/download/v0.1/boston_housing_dataset.zip)
- [Datasets for Data cleaning practise](https://makingnoiseandhearingthings.com/2018/04/19/datasets-for-data-cleaning-practice/)
- [Cleaning up and combining data, a dataset for practice](https://www.r-bloggers.com/cleaning-up-and-combining-data-a-dataset-for-practice/)
- [Datasets from various themes and domains: retail, government. Datasets with a good mix of incorrect, wrongly-input, missing data](https://www.ud-intl.com/dataset)
- [(Specific) Web Traffic Time Series Forecasting](https://www.kaggle.com/c/web-traffic-time-series-forecasting)
- [Quora: What are some dirty/untidy datasets to clean for data analysis? I have just finished datacamp's course on cleaning data using R. I want to practice](https://www.quora.com/What-are-some-dirty-untidy-datasets-to-clean-for-data-analysis-I-have-just-finished-datacamps-course-on-cleaning-data-using-R-I-want-to-practice)
- ["Awesome" Public Datasets](https://github.com/caesar0301/awesome-public-datasets)
- [Bureau of Labor Statistics](https://www.bls.gov/data/)
- [Data.gov](https://www.data.gov/)
- [UK Data Gov](https://data.gov.uk/)
- [HealthData.gov](https://www.healthdata.gov/)
- [Natural Questions](https://ai.google.com/research/NaturalQuestions)
- [NIH Clinical Center CT Images](https://www.nih.gov/news-events/news-releases/nih-clinical-center-releases-dataset-32000-ct-images)
- [Open Data on AWS](https://registry.opendata.aws/)
- [Open Images](https://storage.googleapis.com/openimages/web/index.html)
- [Open ML](https://www.openml.org/search?type=data)
- [Quora list of datasets](https://www.quora.com/Where-can-I-find-large-datasets-open-to-the-public)
- [Stanford Large Network Dataset Collection](https://snap.stanford.edu/data/index.html)
- [Udacity Self-Driving Car dataset](https://github.com/udacity/self-driving-car)
- [USAFACTS](https://usafacts.org/)

## Clean / ready-to-use datasets

- [Boston Housing Dataset (archive contains clean dataset)](https://github.com/neomatrix369/awesome-ai-ml-dl/releases/tag/v0.1) | [Download](https://github.com/neomatrix369/awesome-ai-ml-dl/releases/download/v0.1/boston_housing_dataset.zip)
- Google Dataset Search: 
  - https://toolbox.google.com/datasetsearch
  - https://blog.google/products/search/discovering-millions-datasets-web
  - https://datasetsearch.research.google.com
- [Kaggle Datasets](https://www.kaggle.com/datasets) | Blogs: [1](https://towardsdatascience.com/interesting-datasets-on-kaggle-com-3a4a250b0b85) [2](http://blog.kaggle.com/2016/01/19/introducing-kaggle-datasets/) [3](https://medium.com/@benhamner/introducing-kaggle-datasets-a935f9f76f5) [4](https://stackoverflow.com/questions/52681196/kaggle-datasets-into-jupyter-notebook)
- More Kaggle Datasets
  - [Tweet Sentiment Extraction Extended](https://www.kaggle.com/neomatrix369/tweet-sentiment-extraction-extended)
  - [New York City Taxi Trip Duration Extended](https://www.kaggle.com/neomatrix369/nyc-taxi-trip-duration-extended)
  - [Air Quality Data in India Extended](https://www.kaggle.com/neomatrix369/air-quality-data-in-india-extended)
  - [Google Play Store Apps Extended](https://www.kaggle.com/neomatrix369/google-play-store-apps-extended)
  - [Liverpool Ion Switching competition: 54 features datasets](https://www.kaggle.com/neomatrix369/54-features-datasets-clean-drift-noise-free)
  - [Liverpool Ion Switching competition: Clean datasets (drift & noise removed)](https://www.kaggle.com/neomatrix369/clean-datasets-drift-noise-removed) 
- [Carnegie Mellon University Datasets](http://lib.stat.cmu.edu/datasets/)
- [GeoPlatform Data.gov Search ](https://data.geoplatform.gov/)
- [Data.gov - Data Catalog](https://catalog.data.gov/dataset)
- [TidyTuesday projects on GitHub](https://github.com/rfordatascience/tidytuesday)
- [Enron Email Digest / Fraud detection Dataset](https://www.cs.cmu.edu/~enron/)
- [Mathematics Datasets](https://github.com/deepmind/mathematics_dataset)
- [Data.world datasets](https://data.world)
- [Microsoft Research Open Data](https://msropendata.com/)
- [Free Datasets recommended by r-directory](https://r-dir.com/reference/datasets.html)
- [UC Irvine Machine Learning Repository](https://archive.ics.uci.edu/ml/index.php)
- [Google Research: A large-scale dataset of manually annotated audio events](https://research.google.com/audioset/index.html)
- [Surprising Uses of Synthetic Random Data Sets](https://www.linkedin.com/posts/data-science-central_surprising-uses-of-synthetic-random-data-activity-6612404601515765760-J0AY)
- [How to find datasets for Artificial Intelligence training](https://medium.com/shallow-thoughts-about-deep-learning/how-to-find-datasets-for-artificial-intelligence-9131b2e72e88?fbclid=IwAR1up1xYvKUX4-7DJFs62hTqrfhfLuY9TdNXK56mnmTiUocvv0hgPj6vf4k)
- [Great Github list of public data sets](https://www.linkedin.com/posts/data-science-central_great-github-list-of-public-data-sets-activity-6620739516317646849-YMxO)
- [Wikimedia datasets](https://dumps.wikimedia.org/)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs -](https://www.linkedin.com/posts/philipvollet_machinelearning-datascience-analytics-activity-6715867835287109633-Y_MN)
- [PaySim on kaggle for Money Laundering dataset(s)](https://www.kaggle.com/ntnu-testimon/paysim1)
- [Sklearn provides direct access to openml datasets which hosts around 20,000 datasets and you can access it directly in your python code](https://lnkd.in/g-YYFay) [LinkedIn Post](https://www.linkedin.com/posts/srivatsan-srinivasan-b8131b_datascience-machinelearning-ml-activity-6653512803644768256-w1mM)
- [CNN news Dataset](https://www.tensorflow.org/datasets/catalog/cnn_dailymail)
- [Off-the-Shelf Datasets (licensable datasets to jumpstart your AI projects) (Commercial)](https://appen.com/off-the-shelf-datasets/?utm_source=Web&utm_medium=eblast&mkt_tok=eyJpIjoiWldaak9EUTBObU5oTm1aaiIsInQiOiJKdWlEQVhqclNlcUpNWVhGVW5GT2p2aFpRRjVlZkUyOGZjVHhYSHpsUnNuZkhGVG5rNCtTdm92REdTOXhqTlc0RW1jUlFObnpmM3RaQ3pOZ3RBQ3ArWGZ3RFc4Mk1lRk5FS3d1YklYSnFXeWJROHE2ek9pNW5nU3pCa0gxeExPRCJ9)
- [HuggingFace Datasets](https://huggingface.co/datasets)


## Courses

See [Courses](../courses.md#courses)

## Tools and other resources

- [Measuring dataset similarity using optimal transport](https://www.linkedin.com/posts/philipvollet_machinelearning-datascience-datascientist-activity-6715145819273744384-JghQ)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics - A model-based tool to characterize and diagnose datasets.](https://www.linkedin.com/posts/philipvollet_nlp-machinelearning-ai-activity-6715138376556793856-T5AC)
- [#Texthero is a high-level #Python API to easily and efficiently work with text-based datasets. ](https://www.linkedin.com/posts/inna-vogel-nlp_texthero-text-preprocessing-representation-activity-6689190055614926848-eWF4)

# Contributing

Contributions are very welcome, please share back with the wider community (and get credited for it)!

Please have a look at the [CONTRIBUTING](../CONTRIBUTING.md) guidelines, also have a read about our [licensing](../LICENSE.md) policy.

---

Back to [Data page (table of contents)](README.md)</br>
Back to [main page (table of contents)](../README.md)
