How to show all columns and rows in Pandas

Hi all, in this tutorial, we'll learn how to show all columns and rows in Pandas. The solutions described in the article worked in Jupyter Notebooks and for printing in Python.

Setup

We'll use the following DataFrame from Kaggle - IMDB 5000 Movie Dataset, which can be downloaded and read with Python/Pandas:

import kaggle
import pandas as pd

kaggle.api.authenticate()
kaggle.api.dataset_download_file('carolzhangdc/imdb-5000-movie-dataset', file_name='movie_metadata.csv',  path='data/')

df = pd.read_csv('data/movie_metadata.csv.zip')

DataFrame output in JupyterLab limit the number of shown rows and columns:

On the image we can see all rows and columns displayed by removing default thresholds.

Step 1: Pandas show all columns - max_columns

By default Pandas will display only a limited number of columns. The limit depends on the usage. In this article you can learn more about the limits: How to Show All Columns, Rows and Values in Pandas

To show all columns in Pandas we can set the option: pd.option_context - display.max_columns to None.

with pd.option_context("display.max_columns", None):
    display(df)

This will show all columns in the current DataFrame.

display.max_columns is described as:

In case Python/IPython is running in a terminal this is set to 0 by default and pandas will correctly auto-detect the width of the terminal and switch to a smaller format in case all columns would not fit vertically.

Step 2: Pandas show all rows - max_rows

Pandas will show the first and last rows if the number of the rows is bigger than the threshold. All the other rows will be hidden.

To show all rows in Pandas we can use option - display.max_rows equal to None or some other limit:

with pd.option_context("display.max_rows", None):
    display(df)

The option max_rows is described as:

This sets the maximum number of rows pandas should output when printing out various output. For example, this value determines whether the repr() for a DataFrame prints out fully or just a truncated or summary repr. ‘None’ value means unlimited.

Step 3: Pandas set column width - max_colwidth

Values with length higher than 50 characters will be truncated when we print or display a big DataFrame.

To show full column width in Pandas we can use:

with pd.option_context("display.max_colwidth", None):
    display(df)

max_colwidth

Definition of max_colwidth is:

The maximum width in characters of a column in the repr of a pandas data structure. When the column overflows, a “…” placeholder is embedded in the output. ‘None’ value means unlimited.

Step 4: Pandas show all columns and rows - pd.option_context

Finally if you like to show all rows, columns and values from a Pandas DataFrame we can use the following code to stop Pandas from hiding rows, columns and values:

import pandas as pd
pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.width', None)
pd.set_option('display.max_colwidth', None)

This will be applied for the whole Jupyter Notebook or Python code. So be careful otherwise we may face issues.

Conclusion

We saw how to show all rows, columns and values in Pandas. We've learned how to set options globally and for context.

If you like to learn more about Pandas display option please refer: Pandas Options and settings