Search code examples
pythonpython-polars

Parsing strings with numbers and SI prefixes in polars


Say I have this dataframe:

>>> import polars
>>> df = polars.DataFrame(dict(j=['1.2', '1.2k', '1.2M', '-1.2B']))
>>> df
shape: (4, 1)
┌───────┐
│ j     │
│ ---   │
│ str   │
╞═══════╡
│ 1.2   │
│ 1.2k  │
│ 1.2M  │
│ -1.2B │
└───────┘

How would I go about parsing the above to get:

>>> df = polars.DataFrame(dict(j=[1.2, 1_200, 1_200_000, -1_200_000_000]))
>>> df
shape: (4, 1)
┌───────────┐
│ j         │
│ ---       │
│ f64       │
╞═══════════╡
│ 1.2       │
│ 1200.0    │
│ 1.2e6     │
│ -1.2000e9 │
└───────────┘
>>>

Solution

  • You can use str.extract() and str.strip_chars() to split the parts and then get the resulting number by using Expr.replace() + Expr.pow():

    df.with_columns(
        pl.col('j').str.strip_chars('KMB').cast(pl.Float32) *
        pl.lit(10).pow(
            pl.col('j').str.extract(r'(K|M|B)').replace(['K','M','B'],[3,6,9]).fill_null(0)
        )
    )
    
    ┌─────────────┐
    │ j           │
    │ ---         │
    │ f64         │
    ╞═════════════╡
    │ 1.2         │
    │ 1200.000048 │
    │ 1.2000e6    │
    │ -1.2000e9   │
    └─────────────┘