Getting Public Data Sets for Data Science Projects

14 May 2016

All of us are interested in doing brilliant things with data sets. Most people use Twitter data streams for their projects. But there a lot of free data sets on the Internet. Today, I’m going to list down a few of them. Almost all of these links, I found from a Lynda.com course called Up and Running with Public Data Sets. If you want more details, please watch the complete course on Lynda.com.

  • Quandl (https://www.quandl.com/)
  • Inforum (http://www.inforum.umd.edu/)
  • Google Public Dataset (https://www.google.com/publicdata/directory)
  • Amazon Public Dataset (https://aws.amazon.com/public-data-sets/)
  • US Open Data Portal (https://www.data.gov/)
  • Google Ngram Viewer (https://books.google.com/ngrams)
  • UK Open Data Portal (https://data.gov.uk/)
  • Corpus of Contemporary American English (http://corpus.byu.edu/coca/)
  • World Bank (http://data.worldbank.org/)
  • UN (http://data.un.org/)
  • EuroStat (http://ec.europa.eu/eurostat)
  • CIA World FactBook (https://www.cia.gov/library/publications/the-world-factbook/)
  • American FactFinder (http://factfinder.census.gov/faces/nav/jsf/pages/index.xhtml)
  • US Census Data and Statistics (https://www.usa.gov/statistics)
  • EDGAR (http://www.sec.gov/edgar/searchedgar/companysearch.html)
  • FedStats (https://fedstats.sites.usa.gov/)
  • Singhapore Open Data Poral (https://data.gov.sg/)
  • Dublin Data Portal (http://dublinked.ie/)
  • Ireland Data Portal (https://data.gov.ie/data)
  • Canada Data Portal (http://open.canada.ca/en)

Moreover, Quandl offers R package which you can download data into your R projects very easily. I’m hoping to write a blog post on getting Quandl data with R. This site mainly includes economic data sets.

I’m from Sri Lanka. Sri Lankan researchers might need Sri Lankan data sets as well. Below links will help you to find Sri Lankan data sets.

  • Sri Lanka Open Data Portal (https://www.data.gov.lk)
  • Sri Lanka Police (http://www.police.lk/index.php/information-technology/190)
  • Department of Census and Statistics Sri Lanka (http://www.statistics.gov.lk/)
  • Ministry of Education Statistics (http://www.moe.gov.lk/english/index.php?option=com_content&view=article&id=1220&Itemid=922)

Hope you enjoy playing with those data sets!

Tags

  • data
  • statistics
  • census
  • Data Science