Search code examples
pysparkmultiple-columnsrename

Renaming columns with ' in pyspark

How to rename column "RANDY'S" to 'RANDYS' in pyspark?

I tried below code and its not working

test_rename_df=df.withColumnRenamed('"RANDY''S"','RANDYS')

Note that original column name has double quotes around it

enter image description here

Solution

  • You're adding too many quotes around the original column name. Try this:

    test_rename_df = df.withColumnRenamed("RANDY\'S", "RANDYS")

    Side-note

    When you call df.columns, the column RANDY'S is surrounded by double quotes instead of single quotes to avoid confusion.

    If your column had the name RANDY"S, df.columns would instead use single quotes around the column name (see screenshot below):

    enter image description here