Search code examples
pythonexcelpandascsvexport-to-csv

How can I prevent excel from converting int into scientific notation?


I am pulling data from an API using python and storing it in a pandas dataframe. I export the dataframe in CSV format and when I open it using excel, some columns(having really big numbers) get converted to scientific notation and I don't want that. I want the numbers to stay in an integer format.

I have tried using float format option in the to_csv command and it didn't help either.

This is my export command: df.to_csv("output.csv",index=False, float_format='{:f}'.format, encoding='utf-8',sep=',')

P.s: The Numbers are visible in integer format in the pandas dataframe. Also, when the csv file is opened using notepad, the data is displayed in the proper format.

Current output:

enter image description here

Desired Output:

enter image description here


Solution

  • Excel tends to automatically convert all sorts of things (large integers, alphanumeric codes like 1e2, anything that looks like a date); this can be a problem generally when using it to handle data.

    Two options:

    • Use whole numbers in Excel

      I don't think there's any way to provide whole numbers in CSV for Excel, only decimal numbers (which round off, as you've found) and text (which is difficult for further analysis).

      Excel can store large integers internally (up to 9 quintillion); you should be able to write them directly to .xslx using a library like OpenPyXl.

      However, any formulas based on these numbers will quickly switch to using decimal numbers again, with rounding and loss of precision.

    • Avoid using Excel

      Since you're already using Python, you can continue to use Python for the additional analysis; then you have full control of whether the formulas use integers or decimal numbers at each stage (and, for decimal numbers, how many decimal places).

      The advantage of this is that you have full control; the downside is that you have full control...