Search code examples
dataframedatetimerustrust-polarsrust-chrono

Creating a Datetime Column in Polars DataFrame from i64 Vector in Rust


I'm working with the Polars library in Rust and am struggling to find documentation related to its dtype-datetime feature, especially when compared to its Python counterpart which seems a lot richer with examples.

My goal is to create a dataframe with a column of datetime type. For the purposes of this post, I'll focus only on this column, but please note that my actual dataframe will have multiple columns. I want to use the df! macro with existing Vectors, as shown:

df!(
    "datetime_col" => datetime_vec
)

The datetime_vec would be populated with timestamps, and I've been trying to use the chrono crate (I'm open to suggestions):

let datetime_vec: Vec<i64> = vec![];
...
datetime_vec.push(Utc::now().timestamp_millis());

However, I'm unsure how to utilize this vector with the df! macro such that the column will be recognized as a datetime type. Ideally, it would be done in a way that minimizes allocations and yields best performance.

Has anyone faced this issue? What's the recommended approach? While I did look at Polars' datetime unit tests for clues (linked here), it seems they focus on converting from string to datetime. I need a way to go from i64 to datetime.


Solution

  • Because polars date times are really just ints under the hood, the function you are looking for is cast.

    use polars::prelude::*;
    
    fn main() {
        let v = vec![chrono::Utc::now().timestamp_millis()];
        let mut df = df!["dt" => v].unwrap();
        df.with_column(
            df.column("dt")
                .unwrap()
                .cast(&DataType::Datetime(
                    TimeUnit::Milliseconds,
                    Some("Utc".to_owned()),
                ))
                .unwrap(),
        )
        .unwrap();
        println!("{df:?}");
    }
    
    shape: (1, 1)
    ┌─────────────────────────┐
    │ dt                      │
    │ ---                     │
    │ datetime[ms]            │
    ╞═════════════════════════╡
    │ 2023-08-24 18:27:21.009 │
    └─────────────────────────┘