Having trouble updating code from polars 0.28 to 0.31

Question

Hi I am having trouble updating code from polars 0.28 to 0.31. There are a few calls I use and the problems are not listed in the breaking changes. As there are a few problems to keep the question simple I will only post the first one here and see if that helps with the others.

The first is trying to update the following code. The data frame is a i32 sensor_id a datetime ts and a floating point value column.

 df = df.upsample(&["sensor_id"],
                    "ts", 
                    polars::time::Duration::parse(period),
                    polars::time::Duration::parse("0m")).unwrap();

I get

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: InvalidOperation(ErrString("argument in operation 'upsample' is not explicitly sorted\n\n- If your data is ALREADY sorted, set the sorted flag with: '.set_sorted()'.\n- If your data is NOT sorted, sort the 'expr/series/column' first.\n    "))', /home/glenn/bmos/libbmos/src/lib.rs:4638:58
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I don't see a set_sorted() method in the docs like the error suggests only set_sorted_flag for expr and no idea how to use it.

I have tried something like

df = df.upsample(&[col("sensor_id").set_sorted_flag(polars::series::IsSorted::Ascending)],
                    "ts", 
                    polars::time::Duration::parse(period),
                    polars::time::Duration::parse("0m")).unwrap();

but that gives

the trait bound `polars::prelude::Expr: AsRef<str>` is not satisfied
the following other types implement trait `polars::prelude::IntoVec<T>`:
  <Vec<T> as polars::prelude::IntoVec<T>>
  <bool as polars::prelude::IntoVec<bool>>
required for `&polars::prelude::Expr` to implement `AsRef<str>`
required for `&[polars::prelude::Expr; 1]` to implement `polars::prelude::IntoVec<std::string::String>`

Thanks for any help.

Answer 1

The time column must be sorted as stated here. There are several options:

1. Sort

You can just do the sort:

let df = df.sort(["ts"], false, true).unwrap();

2. set_sorted_flag with LazyFrame

You can use the set_sorted_flag on an expression like this:

let df = df
    .lazy()
    .with_column(col("ts").set_sorted_flag(IsSorted::Ascending))
    .collect()
    .unwrap()
    .upsample(
        &["sensor_id"],
        "ts",
        polars::time::Duration::parse("1d"),
        polars::time::Duration::parse("0m"),
    )
    .unwrap();

3. set_sorted_flag on existing DataFrame

There is another solution without LazyFrame but you have to use unsafe (at least I cannot find another solution):

unsafe {
    let columns = df.get_columns_mut();
    // assuming ts is the first column
    columns[0].set_sorted_flag(IsSorted::Ascending);
}

4. set_sorted_flag before creating DataFrame

When you create the DataFrame with existing Series columns, you can set the flag before creating the DataFrame:

let mut ts = Series::new("ts", <data>);
ts.set_sorted_flag(IsSorted::Ascending);
let df = DataFrame::new(vec![ts, <other columns>]).unwrap();

Warning

When using set_sorted_flag the column must be sorted. Use with care. See here.