Search code examples
httprustgetreqwest

How do you get multiple urls at the same time in a synchronus function


I am getting data from the open weather map API. Currently the data is being retrieved synchronously which is slow. However, the function has to be synchronous as it is part of a library, but it can call an async function. How might I still make concurrent requests to increase performance? A solution that does not use reqwests works, but reqwests is preferred.

fn get_combined_data(open_weather_map_api_url: String, open_weather_map_api_key: String,
                           coordinates: Vec<String>, metric: bool) -> Vec<HashMap<String, String>> {
    let urls: Vec<String> = get_urls(open_weather_map_api_url, open_weather_map_api_key,
                      coordinates.get(0).expect("Improper coordinates").to_string() + "," +
                          coordinates.get(1).expect("Improper coordinates"), metric);
    let mut data: Vec<HashMap<String, String>> = Vec::new();
    for url in urls {
        let request = reqwest::blocking::get(url).expect("Url Get failed").json().expect("json expected");
        data.push(request);
    }
    return data;
}

Solution

  • If your program isn't already async, probably the easiest way might be to use rayon.

    use reqwest;
    use std::collections::HashMap;
    use rayon::prelude::*;
    
    fn get_combined_data(open_weather_map_api_url: String, open_weather_map_api_key: String,
                               coordinates: Vec<String>, metric: bool) -> Vec<HashMap<String, String>> {
    
        let urls: Vec<String> = get_urls(open_weather_map_api_url, open_weather_map_api_key,
                          coordinates.get(0).expect("Improper coordinates").to_string() + "," +
                              coordinates.get(1).expect("Improper coordinates"), metric);
        
        let data : Vec<_>= urls
            .par_iter()
            .map(|&url| reqwest::blocking::get(url).expect("Url Get failed").json().expect("json expected"))
            .collect();
        
        return data;
    }