How to Convert DataFrame to List of Dictionaries in Pandas   

In this quick tutorial, we'll cover how to convert Pandas DataFrame to a list of dictionaries.

Below you can find the quick answer of DataFrame to list of dictionaries:

df.to_dict('records')

Let's explain the solution in a practical example.

Suppose we have DataFrame with data like:

import pandas as pd

df = pd.read_csv('https://raw.githubusercontent.com/softhints/Pandas-Tutorials/master/data/population/population.csv')
df

Convert whole DataFrame to list of dictionaries

Which you would like to convert to list of dictionaries like:

{'Region': 'World',
  '1500': 585,
  '1600': 660,
  '1700': 710,
  '1750': 791,
  '1800': 978,
  '1850': 1262,
  '1900': 1650,
  '1950': 2521,
  '1999': 6008,
  '2008': 6707,
  '2010': 6896,
  '2012': 7052,
  '2050': 9725,
  '2150': 9746},
 {'Region': 'Africa',
  '1500': 86,
  '1600': 114,
  '1700': 106,

In order to achieve this behaviour you can use different approaches. First approach will use Pandas method to_dict('records'):

df.to_dict('records')

This method have several possible options:

• 'dict’ (default) : dict like {column -> {index -> value}}
• 'list’ : dict like {column -> [values]}
• 'series’ : dict like {column -> Series(values)}
• 'split’ : dict like {'index’ -> [index], 'columns’ -> [columns], 'data’ -> [values]}
• 'tight’ : dict like {'index’ -> [index], 'columns’ -> [columns], 'data’ -> [values], 'index_names’ -> [index.names], 'column_names’ -> [column.names]}
• 'records’ : list like [{column -> value}, … , {column -> value}]
• 'index’ : dict like {index -> {column -> value}}

Convert columns to list of dictionaries

If you want to convert only some columns to a list of dictionaries you can use similar syntax. This can be achieved by selecting the columns and then applying method .to_dict('records'):

df[['Region', '1500', '1600', '1700']].to_dict('records')

Result is subset of the selected columns:

[{'Region': 'World', '1500': 585, '1600': 660, '1700': 710},
 {'Region': 'Africa', '1500': 86, '1600': 114, '1700': 106},
 {'Region': 'Asia', '1500': 282, '1600': 350, '1700': 411},
 {'Region': 'Europe', '1500': 168, '1600': 170, '1700': 178},
 {'Region': 'Latin America [Note 1] \u200b',
  '1500': 40,
  '1600': 20,
  '1700': 10},
 {'Region': 'Northern America [Note 1] \u200b',
  '1500': 6,
  '1600': 3,
  '1700': 2},
 {'Region': 'Oceania', '1500': 3, '1600': 3, '1700': 3}]

Transformation is visible from the image below:

Convert DataFrame to list of dictionaries - column wise

What if you like to get list of dictionaries column wise like:

{'Region': {0: 'World',
  1: 'Africa',
  2: 'Asia',
  3: 'Europe',
  4: 'Latin America [Note 1] \u200b',
  5: 'Northern America [Note 1] \u200b',
  6: 'Oceania'},
 '1500': {0: 585, 1: 86, 2: 282, 3: 168, 4: 40, 5: 6, 6: 3},
 '1600': {0: 660, 1: 114, 2: 350, 3: 170, 4: 20, 5: 3, 6: 3},
 '1700': {0: 710, 1: 106, 2: 411, 3: 178, 4: 10, 5: 2, 6: 3},

You can achieve this transformation by transposing the DataFrame with .T and using option index:

df.T.to_dict('index')

Custom Conversion of DataFrame to list of dictionaries

Finally lets cover the case when there is a custom logic for transformation of DataFrame to list of dictionaries in Pandas. This option is a bit slower than the rest and works fine for small and medium sized data.

We are going to iterate over all rows by:

data_dict = []
for index, row in df[['Region', '1500', '1600', '1700']].iterrows():
    data_dict.append({
            'Region': row['Region'],
            '1500': row['1500'],
            '1600': row['1600'],
            '1700': row['1700'],
            })

the result would be:

[{'Region': 'World', '1500': 585, '1600': 660, '1700': 710},
 {'Region': 'Africa', '1500': 86, '1600': 114, '1700': 106},
 {'Region': 'Asia', '1500': 282, '1600': 350, '1700': 411},
 {'Region': 'Europe', '1500': 168, '1600': 170, '1700': 178},
 {'Region': 'Latin America [Note 1] \u200b',
  '1500': 40,
  '1600': 20,
  '1700': 10},
 {'Region': 'Northern America [Note 1] \u200b',
  '1500': 6,
  '1600': 3,
  '1700': 2},
 {'Region': 'Oceania', '1500': 3, '1600': 3, '1700': 3}]

To convert all columns to list of dicts with custom logic you can use code like:

data_dict = []
for index, row in df[df.columns].iterrows():
    data_dict.append({
            row.to_list()[0] : row.to_list()[1:]
            })
data_dict    

result:

[{'World': [585,
   660,
   710,
   791,
   978,
   1262,
   1650,
   2521,
   6008,
   6707,
   6896,
   7052,
   9725,
   9746]},
 {'Africa': [86,
   114,
   106,
   106,
   107,

Resources

• Notebook
• pandas.DataFrame.to_dict