> ## Content Index
> Fetch the complete content index at: https://datascientyst.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# List of Aggregation Functions(aggfunc) for GroupBy in Pandas
- URL: https://datascientyst.com/list-aggregation-functions-aggfunc-groupby-pandas/
- Published: 2021-08-29T22:14:48.000Z
- Updated: 2023-04-29T06:43:57.000Z
- Author: John D K
- Tags: aggfunc

In this article, you can find the list of the **available aggregation functions for groupby in Pandas**:

- `count` / `nunique` – non-null values / count number of unique values
- `min` / `max` – minimum/maximum
- `first` / `last` \- return first or last value per group
- `unique` \- all unique values from the group
- `std` – standard deviation
- `sum` – sum of values
- `mean` / `median` / `mode` – mean/median/mode
- `var` \- unbiased variance
- `mad` \- mean absolute deviation
- `skew` \- unbiased skew
- `sem` \- standard error of the mean
- `quantile`

Those functions can be used with `groupby` in order to return statistical information about the groups.

In the next section we will cover all aggregation functions with simple examples.

![pandas-aggfunc-stats-functions-min-max-group](https://datascientyst.com/content/images/2022/10/pandas-aggfunc-stats-functions-min-max-group.opti.webp)

## Step 1: Create DataFrame for aggfunc

Let us use the earthquake dataset. We are going to create new column `year_month` and `groupby` by it:

```python
import pandas as pd

df = pd.read_csv(f'../data/earthquakes_1965_2016_database.csv.zip')
cols = ['Date', 'Time', 'Latitude', 'Longitude', 'Depth', 'Magnitude Type', 'Type', 'ID']
df = df[cols]

```

result:

| Date       | Latitude | Longitude | Depth | Type       |
| ---------- | -------- | --------- | ----- | ---------- |
| 01/02/1965 | 19.246   | 145.616   | 131.6 | Earthquake |
| 01/04/1965 | 1.863    | 127.352   | 80.0  | Earthquake |
| 01/05/1965 | \-20.579 | \-173.972 | 20.0  | Earthquake |
| 01/08/1965 | \-59.076 | \-23.557  | 15.0  | Earthquake |
| 01/09/1965 | 11.938   | 126.427   | 15.0  | Earthquake |

Next we are going to create new column with information - combination of the year and the month:

```python
df['Date'] = pd.to_datetime(df['Date'], utc=True)
df['year_month'] = df['Date'].dt.to_period('M')

```

## Step 2: Pandas describe DataFrame

In each step we will see examples of using each of the aggregating functions associated with Pandas groupby function.

We will start with the method `describe`. This method returns basic information about the column:

```python
df.groupby('year_month')['Depth'].describe()

```

`describe` returns multiple aggfunc-s like: count, mean, std, min, max:

| count | mean       | std        | min  | 25%    | 50%  | 75%     | max   |
| ----- | ---------- | ---------- | ---- | ------ | ---- | ------- | ----- |
| 13.0  | 101.115385 | 152.237697 | 15.0 | 20.000 | 35.0 | 95.000  | 565.0 |
| 54.0  | 47.712963  | 80.122216  | 10.0 | 20.075 | 25.1 | 34.375  | 482.9 |
| 38.0  | 62.055263  | 97.890288  | 10.0 | 25.000 | 31.3 | 47.850  | 560.8 |
| 33.0  | 112.163636 | 183.812083 | 10.0 | 25.000 | 30.7 | 64.700  | 635.0 |
| 22.0  | 80.972727  | 114.894839 | 10.0 | 25.875 | 35.0 | 105.000 | 553.8 |

![](https://datascientyst.com/content/images/2021/08/list-aggregation-functions-aggfunc-groupby-pandas.png)

## Step 3: Pandas all aggfunc for DataFrame

In this step you can find examples for all aggfunc-s applied on a DataFrame. The list of the functions is below.

Note that by default method `groupby` will exclude all `NaN` values. In order to change this behavior you can use parameter - `dropna=False`

```python
aggfuncs = [ 'count', 'sum', 'sem', 'skew', 'mean', 'min', 'max', 
'std', 'quantile', 'nunique', 'mad', 'size', pd.Series.mode, 'var', 'unique']
df.groupby('year_month', dropna=False)['Depth'].agg(aggfuncs)

```

result:

| year\_month | 1965-01                                                     | 1965-02                                                                                                                                                                                     |
| ----------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| count       | 13                                                          | 54                                                                                                                                                                                          |
| sum         | 1314.5                                                      | 2576.5                                                                                                                                                                                      |
| sem         | 42.22314                                                    | 10.903253                                                                                                                                                                                   |
| skew        | 2.744523                                                    | 4.256526                                                                                                                                                                                    |
| mean        | 101.115385                                                  | 47.712963                                                                                                                                                                                   |
| min         | 15.0                                                        | 10.0                                                                                                                                                                                        |
| max         | 565.0                                                       | 482.9                                                                                                                                                                                       |
| std         | 152.237697                                                  | 80.122216                                                                                                                                                                                   |
| quantile    | 35.0                                                        | 25.1                                                                                                                                                                                        |
| nunique     | 9                                                           | 30                                                                                                                                                                                          |
| mad         | 95.56213                                                    | 39.163306                                                                                                                                                                                   |
| size        | 13                                                          | 54                                                                                                                                                                                          |
| mode        | 20.0                                                        | 25.0                                                                                                                                                                                        |
| var         | 23176.31641                                                 | 6419.569451                                                                                                                                                                                 |
| unique      | \[131.6, 80.0, 20.0, 15.0, 35.0, 95.0, 565.0, 227.9, 55.0\] | \[482.9, 15.0, 10.0, 30.3, 30.0, 25.0, 20.0, 24.0, 31.8, 39.5, 30.4, 17.8, 27.7, 30.1, 37.4, 17.5, 22.5, 25.2, 17.7, 32.5, 200.0, 100.0, 53.5, 340.0, 55.0, 35.0, 40.4, 40.0, 20.3, 160.0\] |

## Step 4: Pandas aggfunc - Count, Nunique, Size, Unique

In this step we will cover 4 aggregation functions:

- `count` \- compute count of group, excluding missing values
- `size` \- compute group sizes
- `unique` \- return unique values
- `nunique` \- return number of unique elements in the group.

Example of using the functions and the result:

```python
aggfuncs = [ 'count', 'size', 'nunique', 'unique']
df.groupby('year_month')['Depth'].agg(aggfuncs)

```

output:

|             | count | size | nunique | unique                                                                                                                                                                                      |
| ----------- | ----- | ---- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| year\_month |       |      |         |                                                                                                                                                                                             |
| 1965-01     | 13    | 13   | 9       | \[131.6, 80.0, 20.0, 15.0, 35.0, 95.0, 565.0, 227.9, 55.0\]                                                                                                                                 |
| 1965-02     | 54    | 54   | 30      | \[482.9, 15.0, 10.0, 30.3, 30.0, 25.0, 20.0, 24.0, 31.8, 39.5, 30.4, 17.8, 27.7, 30.1, 37.4, 17.5, 22.5, 25.2, 17.7, 32.5, 200.0, 100.0, 53.5, 340.0, 55.0, 35.0, 40.4, 40.0, 20.3, 160.0\] |
| 1965-03     | 38    | 38   | 24      | \[30.0, 40.0, 33.6, 105.2, 10.0, 15.0, 14.8, 25.7, 200.0, 35.0, 560.8, 45.0, 50.0, 20.0, 207.8, 32.1, 25.0, 224.9, 28.9, 30.5, 48.8, 55.0, 70.0, 75.0\]                                     |
| 1965-04     | 33    | 33   | 22      | \[60.0, 10.0, 30.7, 25.0, 20.0, 39.2, 50.0, 65.0, 17.5, 543.7, 635.0, 570.0, 421.7, 165.0, 15.0, 35.0, 70.0, 30.0, 36.8, 37.1, 64.7, 480.0\]                                                |
| 1965-05     | 22    | 22   | 16      | \[15.0, 22.5, 90.0, 110.0, 10.0, 50.0, 153.1, 25.0, 35.0, 68.4, 120.0, 553.8, 28.5, 30.1, 125.0, 150.0\]                                                                                    |

## Step 5: Pandas aggfunc - First and Last

There are two functions which can return the first or the last value of the group. They are:

- `first` \- compute first of group values
- `last` \- compute first of group values

Example of their usage:

```python
aggfuncs = [ 'first', 'last']
df.groupby('year_month')['Depth'].agg(aggfuncs)

```

result:

|             | first | last  |
| ----------- | ----- | ----- |
| year\_month |       |       |
| 1965-01     | 131.6 | 55.0  |
| 1965-02     | 482.9 | 10.0  |
| 1965-03     | 30.0  | 75.0  |
| 1965-04     | 60.0  | 480.0 |
| 1965-05     | 15.0  | 150.0 |

## Step 6: Pandas aggfunc - Sum, Min, Max

For numeric or datetime columns we can get the minimum, maximum or the sum by those aggfunc-s:

- `sum` \- compute sum of group values
- `min` \- compute min of group values
- `max` \- compute max of group values

How to get the sum, maximum and the minimum per group:

```python
aggfuncs = [ 'sum', 'min', 'max']
df.groupby('year_month')['Depth'].agg(aggfuncs)

```

the result is:

|             | sum    | min  | max   |
| ----------- | ------ | ---- | ----- |
| year\_month |        |      |       |
| 1965-01     | 1314.5 | 15.0 | 565.0 |
| 1965-02     | 2576.5 | 10.0 | 482.9 |
| 1965-03     | 2358.1 | 10.0 | 560.8 |
| 1965-04     | 3701.4 | 10.0 | 635.0 |
| 1965-05     | 1781.4 | 10.0 | 553.8 |

## Step 7: Pandas aggfunc - Mean, Median, Mode

There are several very important statistics which are:

- The mean is the average of a group values
- The mode is the most common number in a group
- The median is the middle of the group values

They are implemented in Pandas as functions:

- `mean` \- compute mean of groups, excluding missing values
- `pd.Series.mode` \- return the mode(s) of the Series.
- `median` \- compute median of groups, excluding missing values.

They can be compute on Pandas `groupby` object by next syntax:

```python
aggfuncs = [ 'mean', 'median', pd.Series.mode]
df.groupby('year_month')['Depth'].agg(aggfuncs)

```

result:

|             | mean       | median | mode |
| ----------- | ---------- | ------ | ---- |
| year\_month |            |        |      |
| 1965-01     | 101.115385 | 35.0   | 20.0 |
| 1965-02     | 47.712963  | 25.1   | 25.0 |
| 1965-03     | 62.055263  | 31.3   | 30.0 |
| 1965-04     | 112.163636 | 30.7   | 25.0 |
| 1965-05     | 80.972727  | 35.0   | 35.0 |

## Step 7: Pandas aggfunc - STD, MAD, Var

Another important methods in statistics are:

- `std` \- compute **standard deviation of groups**, excluding missing value
- `var` \- compute **variance of groups**, excluding missing values
- `mad` \- return the **mean absolute deviation** of the values over the requested axis

How to calculate the standard deviation, variance and mean absolute deviation of groups:

```python
aggfuncs = ['mad', 'std', 'var']
df.groupby('year_month')['Depth'].agg(aggfuncs)

```

output:

|             | mad        | std        | var          |
| ----------- | ---------- | ---------- | ------------ |
| year\_month |            |            |              |
| 1965-01     | 95.562130  | 152.237697 | 23176.316410 |
| 1965-02     | 39.163306  | 80.122216  | 6419.569451  |
| 1965-03     | 53.121745  | 97.890288  | 9582.508485  |
| 1965-04     | 129.843526 | 183.812083 | 33786.881761 |
| 1965-05     | 66.826446  | 114.894839 | 13200.823983 |

## Step 7: Pandas aggfunc - Skew, Sem, quantile

Let's check few other functions which are not very popular like:

- `skew` \- return unbiased skew over requested axis
- `sem` \- compute standard error of the mean of groups, excluding missing values
- `quantile` \- return group values at the given quantile, a la numpy.percentile

```python
aggfuncs = [ 'skew', 'sem', 'quantile']
df.groupby('year_month')['Depth'].agg(aggfuncs)

```

result:

|             | skew     | sem       | quantile |
| ----------- | -------- | --------- | -------- |
| year\_month |          |           |          |
| 1965-01     | 2.744523 | 42.223140 | 35.0     |
| 1965-02     | 4.256526 | 10.903253 | 25.1     |
| 1965-03     | 4.036620 | 15.879902 | 31.3     |
| 1965-04     | 2.054374 | 31.997576 | 30.7     |
| 1965-05     | 3.604639 | 24.495662 | 35.0     |

## Step 8: User defined aggfunc for groupby

It's possible in Pandas to define your own aggfunc and use it with a groupby method.

In the next example we will define a function which will compute the NaN values in each group:

```python
def countna(x):
    return (x.isna()).sum()

df.groupby('year_month')['Depth'].agg([countna])

```

result:

|             | countna |
| ----------- | ------- |
| year\_month |         |
| 1965-01     | 0       |
| 1965-02     | 0       |
| 1965-03     | 0       |
| 1965-04     | 0       |
| 1965-05     | 0       |

## Step 9: Pandas aggfuncs from scipy or numpy

Finally let's check how to use aggregation functions with `groupby` from `scipy` or `numpy`

Below you can find a `scipy` example applied on Pandas `groupby` object:

```python
from scipy import stats

df.groupby('year_month')['Depth'].agg(lambda x: stats.mode(x)[0])

```

result:

```
year_month
1965-01    20.0
1965-02    25.0
1965-03    30.0
1965-04    25.0

```

Example for `numpy.count_nonzero` method used with Pandas `groupby` method:

```python
import numpy as np

df.groupby('year_month')['Depth'].agg(np.count_nonzero)

```

Output:

```
year_month
1965-01    13
1965-02    54
1965-03    38
1965-04    33

```

## Resources

- [Notebook](https://github.com/softhints/datascientyst/blob/master/aggfunc/1.list-aggregation-functions-aggfunc-groupby-pandas.ipynb?ref=datascientyst.com)
- [pandas.core.groupby.DataFrameGroupBy.describe](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.describe.html?ref=datascientyst.com)
- [pandas.core.groupby.DataFrameGroupBy.count](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.count.html?ref=datascientyst.com)
- [pandas.core.groupby.DataFrameGroupBy.size](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.size.html?ref=datascientyst.com)
- [pandas.core.groupby.DataFrameGroupBy.nunique](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.nunique.html?ref=datascientyst.com)
- [pandas.core.groupby.SeriesGroupBy.unique](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.SeriesGroupBy.unique.html?ref=datascientyst.com)
- [pandas.core.groupby.GroupBy.first](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.first.html?ref=datascientyst.com)
- [pandas.core.groupby.GroupBy.last](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.last.html?ref=datascientyst.com)
- [pandas.core.groupby.GroupBy.sum](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.sum.html?ref=datascientyst.com)
- [pandas.core.groupby.GroupBy.min](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.min.html?ref=datascientyst.com)
- [pandas.core.groupby.GroupBy.max](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.max.html?ref=datascientyst.com)
- [pandas.core.groupby.GroupBy.mean](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.mean.html?ref=datascientyst.com)
- [pandas.core.groupby.GroupBy.median](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.median.html?ref=datascientyst.com)
- [pandas.Series.mode](https://pandas.pydata.org/docs/reference/api/pandas.Series.mode.html?ref=datascientyst.com)
- [pandas.core.groupby.GroupBy.std](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.std.html?ref=datascientyst.com)
- [pandas.core.groupby.GroupBy.var](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.var.html?ref=datascientyst.com)
- [pandas.core.groupby.DataFrameGroupBy.mad](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.mad.html?ref=datascientyst.com)
- [pandas.core.groupby.DataFrameGroupBy.quantile](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.quantile.html?ref=datascientyst.com)
- [pandas.core.groupby.DataFrameGroupBy.skew](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.skew.html?ref=datascientyst.com)
- [pandas.core.groupby.GroupBy.sem](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.sem.html?ref=datascientyst.com)
- [pandas.DataFrame.max](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.max.html?ref=datascientyst.com)
- [pandas.DataFrame.min](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.min.html?ref=datascientyst.com)
- [pandas.DataFrame.median](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.median.html?ref=datascientyst.com)
- [pandas.DataFrame.mean](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.mean.html?ref=datascientyst.com)
- [pandas.DataFrame.sum](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.sum.html?ref=datascientyst.com)
- [pandas.DataFrame.count](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.count.html?ref=datascientyst.com)
- [pandas.DataFrame.std](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.std.html?ref=datascientyst.com)
- [pandas.DataFrame.mode](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.mode.html?ref=datascientyst.com)
- [pandas.DataFrame.var](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.var.html?ref=datascientyst.com)