How to untruncate the print output. The pd.set_option() option does not work

Question

I am using pandas 1.4.4 and Python 3.9.13 in Jupyter notebook and trying to make my print output appear not truncated.

My code is below:

for column in categorical_cols[1:]:
    unique_values = df[column].value_counts()
    non_null_values = df[column].count()
    print(f'Column: {column} - Non value values: {non_null_values}')
    print(unique_values)
    print()

Basically categorical_cols is the dataframe containing columns with non numerical values. There is a code for it below.

categorical_cols = df.select_dtypes(include=['object']).columns

And with for loop I am trying to find out unique values in each categorical column, which works fine. The only problem is that my output in Jupyter is truncated. Here is a print screen

Some info about df: Int64Index: 34434 entries, 0 to 34433 Data columns (total 51 columns)

I have tried various pandas.set_option below, but they do not seem to be working:

• pd.set_option('display.max_rows', None)
• pd.set_option('display.max_columns', None)
• pd.set_option('display.pprint_nest_depth', 10)
• pd.set_option('display.large_repr', 'info')

Answer 1

The screenshot that has been added indicates that by 'truncated' you seem to mean that 'scrolled' view has been activated.

See this post and the answers to get an idea of what is going on here and how you can change it. See bottom of this answer for more about this mode of view that gets activated in classic notebook if you print a lot of output.

This happens when you print a lot of output to your cell. I speculate it may be an old, cautious way built in to Jupyter to better handle a lot of output when computers weren't generally as powerful. Note that this doesn't happen in modern JupyterLab, and so one solution is to start using the more modern offering of JupyterLab. (I don't know what Jupyter Notebook Version 7 or higher does. I'm assuming that nbclassic is what you are using. See here if 'nbclassic' and 'Jupyter Notebook Version 7' don't make sense to you.

Other solutions if you want to keep using nbclassic:

• Toggle that mode off by clicking on the left hand side of the view cell.
• Based on using IPython/Jupyter's %%capture and %store magic illustrated here you can add %%capture out to the top of your cell with a lot of output. Then in the next cell run %store out.stdout >my_ton_of_text.txt to save a file that has all the output then you can open that text file in Jupyter or your own text editor to peruse it.
• Edit your cell's code to use Python to accumulate a string instead of printing it and then use %store magic covered above to save it as a file.

---

Details on scrolled view

If I ran the following code, I'll get the mode you show in your screenshot. If I run this code in a new notebook it will happen because there is a lot of output:

for x in range(2000):
    print(x)

So if I save the notebook file and then open it in a text editor then I see the following among the code for the .ipynb.

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e0c96475",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",

Note the "scrolled": true in the metadata for it.
You can cycle through the different view options by clicking on the left side of the cell output area. One of the views is it off.