Search code examples
pythonpandasdataframesearchreplace

Pandas: Search and replace a string across all columns in a dataframe


I'm trying to search for a string 'NONE' that is in uppercase across all columns in a dataframe and replace it with 'None'. If it is already 'None' I don't do anything.

I tried using the lambda function but its not working.

ps: In the data None is not a keyword. It is a string eg: "None"

df=data.apply(lambda row: 'None' if row.astype(str).str.contains('NONE').any() else row, axis=1)

Sample input

   A      B
None    234
NONE   NONE
565     347

Expected Output

   A     B
None   234
None  None
565    347

Solution

  • A straightforward call to replace will do the job because the string being replaced is the entire value in a given location, so there's no need to do a complicated contains search.

    Also, unlike str.replace, replace is also a DataFrame method, so it will search through all values in the dataframe.

    df = pd.DataFrame({'A': ['None', 'NONE', '565'], 
                       'B': ['234', 'NONE', '347']})
    # replace NONE by None
    df = df.replace('NONE', 'None')
    print(df)
    
          A     B
    0  None   234
    1  None  None
    2   565   347
    

    We can also pass regex=True to replace values using a regex pattern:

    df = df.replace('NONE', 'None', regex=True)
    

    This would be useful if the value to be replaced is part of a longer string.