How to Group By Multiple Columns in Pandas

To group by multiple columns in Pandas DataFrame can we use the method groupby()?

We will cover:

• group by multiple columns
• group by several statistical functions
• named group by
• Series vs DataFrame group by

To group by multiple columns and using several statistical functions we are going to use next functions:

• groupby()
• agg()
• 'mean', 'count', 'sum'

df.groupby(['publication', 'date_m']).agg(['mean', 'count', 'sum'])

Let's see all the steps in order to find the statistics for each group.

Step 1: Create sample DataFrame

Let's use the following Kaggle Dataset:

You can find the sample data from the repository of the notebook or use the link below to download it.

To learn more about reading Kaggle data with Python and Pandas: How to Search and Download Kaggle Dataset to Pandas DataFrame

Step 2: Group by multiple columns

First lets see how to group by a single column in a Pandas DataFrame you can use the next syntax:

df.groupby(['publication'])

In order to group by multiple columns we need to give a list of the columns.

Group by two columns in Pandas:

df.groupby(['publication', 'date_m'])

The columns and aggregation functions should be provided as a list to the groupby method.

Step 3: GroupBy SeriesGroupBy vs DataFrameGroupBy

The object returned after the groupby of multiple columns depends on the usage of the groups. Let's check it by examples:

df.groupby('publication')

returns:

pandas.core.groupby.generic.DataFrameGroupBy

df.groupby(['publication'])['url']

returns:

pandas.core.groupby.generic.SeriesGroupBy

df.groupby(['publication'])[['url', 'date']]

returns:

pandas.core.groupby.generic.DataFrameGroupBy

If you use a single column after the groupby you will get SeriesGroupBy otherwise you will have DataFrameGroupBy.

Step 4: Apply multiple statistical functions

Applying multiple aggregation functions to a groupby is done by method: agg.

Note: that another function aggregate exists which and agg is an alias for it.

The functions can be passed as a list.

The available aggregation functions for group by in Pandas are:

• count – non-null values
• min / `max – minimum/maximum
• std – standard deviation
• sum – sum of values
• mean / median – mean/median
• mode
• var

Below you can find the syntax in order to apply multiple statistical functions on multiple pandas columns while grouping by multiple columns:

df.groupby(['publication', 'date_m'])[['claps', 'reading_time']].agg(['mean', 'count', 'sum'])

result:

If you don't provide columns after the grouper than all numeric/datetime columns will be returned:

df.groupby(['publication', 'date_m']).agg(['mean'])

all columns on which agg functions can be applied are returned:

Note that the above results have MultiIndex. In order to work with MultiIndex you can check those articles:

• How to Sort MultiIndex in Pandas
• What is a DataFrame MultiIndex in Pandas

Step 5: Pandas groupby and named aggregations

What if you like to group by multiple columns with several aggregation functions and would like to have - named aggregations.

In other words you need to get the next information:

• id - mean of this column
• claps - mean, count and range

You can also name the columns to meet your needs. Finally you can get them without MultiIndex. This is possible by next syntax:

df.groupby('publication').agg(
             id_mean = ('id', 'mean'),
             claps_mean = ('claps', 'mean'),
             claps_count = ('claps', 'count'),
             claps_range = ('claps', lambda x: x.max() - x.min()))

this will return:

Conclusion

In this article we saw** how to group by multiple columns in Pandas.**

We saw how to group by two columns and use different aggregation functions in Pandas DataFrame.

Resources

• Notebook
• pandas.DataFrame.agg
• named-aggregation