How to Search and Download Kaggle Dataset to Pandas DataFrame   

In this post, we'll take a brief look at the Kaggle Datasets and how to download/import them with Python. By the end, we'll see how to list, download single or multiple datasets and finally how to read them into Pandas DataFrame.

Step 1: Create Kaggle API token

First you will need to visit: Kaggle and create a new account. You can sign up with your google account.

In order to create new Kaggle API token follow:

• Open your profile picture(top right)
• Account - the url is: https://www.kaggle.com/<username>/account
• API
• Create new API Token
• This will generate kaggle.json file
• Place the file in your home folder as: ~/.kaggle/kaggle.json
• For more security (optional) - chmod 600 ~/.kaggle/kaggle.json

More info is available on this link: Kaggle API

Step 2: Install Python's package for Kaggle

Next we are going to install the package which is going to download the datasets from Kaggle. You can install kaggle package in virtual environment by:

pip install kaggle

or for the user:

pip install --user kaggle

Step 3: Download single file from Kaggle dataset

Now we are going to demonstrate how to download a single CSV file from the Kaggle dataset. This will work only if previous steps were done successfully:

import kaggle

kaggle.api.authenticate()
kaggle.api.dataset_download_file('dorianlazar/medium-articles-dataset', file_name='medium_data.csv',  path='data/')

In the example above we are going to download file: medium_data.csv from: dorianlazar/medium-articles-dataset.

The file will be downloaded in the folder data/.

The file can be read by:

import pandas as pd
pd.read_csv('data/medium_data.csv.zip')

which produce:

Step 4: Download multiple files from Kaggle dataset

If we like to get all files from a Kaggle dataset then we can get them by:

import kaggle

kaggle.api.authenticate()

kaggle.api.dataset_download_files('dorianlazar/medium-articles-dataset', path='data/')

Note that this might be pretty slow for big datasets. We are downloading all files from the dataset mentioned above.

Step 5: List and search Kaggle datasets with API

Finally let's find how to list and search for Kaggle datasets. This can be done by next command:

!kaggle datasets list -s article

Where we are searching for the keyword - article. The output is: