In Rust, using Polars, I am writing a custom function to be used within apply/map. This works well:
fn capita(x: Series) -> Result<Series> {
let y = x
.utf8()
.unwrap()
.par_iter() //ParallelIterator
However, if my Series was of type f64, then this doesn't work:
fn other_fn(x: Series) -> Result<Series> {
let y = x
.f64()
.unwrap()
.par_iter() // <--- no method par_iter found for reference ChunkedArray<Float64Type>
Is there a possible workaround? Looking at utf8.rs I need something like that but for Float64Chunked type.
Many thanks
EDIT I think this workaround has worked, although turned out to be slower, AND giving an unexpected result. using par_bridge:
pub fn pa_fa(s: &mut [Series])->Result<Series>{
let u = s[2].f64()?;
let n = s[1].f64()?;
let n_iter = n.into_iter();
let c: Vec<f64> = n_iter.zip(u).par_bridge().map(
// ignore this line let c: Vec<f64> = n_iter.zip(u).map(
|(n,u)|{
n.unwrap().powf(1.777)*u.unwrap().sqrt()
}
).collect();
Ok(Series::new("Ant", c))
The alternative I found was to collect ChunkedArray into a Vector:
use polars::prelude::*;
use rayon::prelude::*;
fn round_series_float64(series: Series, decimals: u32) -> PolarsResult<Series> {
let chunkedarray_floatf64 = series
.f64()?
.into_iter()
.collect::<Float64Chunked>();
let vec_opt_f64: Vec<Option<f64>> = chunkedarray_floatf64
.into_iter()
.collect();
// No sorting required as rayon collects already sorted!
let new_series: Series = vec_opt_f64
//.into_iter()
.into_par_iter() // rayon: parallel iterator
.map(|opt_f64: Option<f64>|
opt_f64.map(|float64| round_f64(float64, decimals))
)
.collect::<Float64Chunked>()
.into_series();
/*
// No use of rayon:
let new_series: Series = series
.f64()?
.into_iter()
.map(|opt_f64: Option<f64>|
opt_f64.map(|float64| round_f64(float64, decimals))
)
.collect::<Float64Chunked>()
.into_series();
*/
Ok(new_series)
}
Such that:
pub fn round_f64(value: f64, decimals: u32) -> f64 {
if decimals == 0 {
value.round()
} else {
let multiplier = 10.0_f64.powf(decimals as f64);
(value * multiplier).round() / multiplier
}
}