How to combine multiple web requests into one query/table?

Question

I want to create a Canadian holiday calendar for multiple years. For this, I'm using canada-holidays.ca/api. The thing is, if I want multiple years, I need to create a request for each year and then append them but doing so create a lot of requests that I don't want or need.

That's why I'm trying to integrate multiple source at the same time in a single query in Power Query.

I found this post as an inspiration for what I want to do but it is based on Excel so it needs to be adjusted a bit.

That's why, I wrote this code :

let
//2021 Holidays
    Source1 = Json.Document(Web.Contents("https://canada-holidays.ca/api/v1/holidays?year=2021")),
    #"Converti en table1" = Table.FromRecords({Source1}),
    #"ExpandList1" = Table.ExpandListColumn(#"Converti en table1", "holidays"),
    #"ExpandRecord1" = Table.ExpandRecordColumn(#"ExpandList1", "holidays", {
        "id", 
        "date", 
        "nameEn", 
        "nameFr", 
        "federal", 
        "observedDate"}, {
            "id", 
            "date", 
            "nameEn", 
            "nameFr", 
            "federal", 
            "observedDate"}),
    #"Type modifié1" = Table.TransformColumnTypes(#"ExpandRecord1",{
        {"id", Int64.Type}, 
        {"date", type date}, 
        {"nameEn", type text}, 
        {"nameFr", type text}, 
        {"federal", Int64.Type}, 
        {"observedDate", type date}}),
    #"Lignes filtrées1" = Table.SelectRows(#"Type modifié1", each (
        [nameEn] = "Boxing Day" or 
        [nameEn] = "Canada Day" or 
        [nameEn] = "Christmas Day" or 
        [nameEn] = "Day of Mourning for Queen Elizabeth II" or 
        [nameEn] = "Easter Monday" or 
        [nameEn] = "Good Friday" or 
        [nameEn] = "Labour Day" or 
        [nameEn] = "National Day for Truth and Reconciliation" or 
        [nameEn] = "New Year’s Day" or 
        [nameEn] = "Remembrance Day" or 
        [nameEn] = "Saint-Jean-Baptiste Day" or 
        [nameEn] = "Thanksgiving" or 
        [nameEn] = "Victoria Day")),

//2022 Holidays
    Source2 = Json.Document(Web.Contents("https://canada-holidays.ca/api/v1/holidays?year=2022")),
    #"Converti en table2" = Table.FromRecords({Source2}),
    #"ExpandList2" = Table.ExpandListColumn(#"Converti en table2", "holidays"),
    #"ExpandRecord2" = Table.ExpandRecordColumn(#"ExpandList2", "holidays", {
        "id", 
        "date", 
        "nameEn", 
        "nameFr", 
        "federal", 
        "observedDate"}, {
            "id", 
            "date", 
            "nameEn", 
            "nameFr", 
            "federal", 
            "observedDate"}),
    #"Type modifié2" = Table.TransformColumnTypes(#"ExpandRecord2",{
        {"id", Int64.Type}, 
        {"date", type date}, 
        {"nameEn", type text}, 
        {"nameFr", type text}, 
        {"federal", Int64.Type}, 
        {"observedDate", type date}}),
    #"Lignes filtrées2" = Table.SelectRows(#"Type modifié2", each (
        [nameEn] = "Boxing Day" or 
        [nameEn] = "Canada Day" or 
        [nameEn] = "Christmas Day" or 
        [nameEn] = "Day of Mourning for Queen Elizabeth II" or 
        [nameEn] = "Easter Monday" or 
        [nameEn] = "Good Friday" or 
        [nameEn] = "Labour Day" or 
        [nameEn] = "National Day for Truth and Reconciliation" or 
        [nameEn] = "New Year’s Day" or 
        [nameEn] = "Remembrance Day" or 
        [nameEn] = "Saint-Jean-Baptiste Day" or 
        [nameEn] = "Thanksgiving" or 
        [nameEn] = "Victoria Day")),

    #"TableAppend" = Table.Combine({#"Lignes filtrées1"}, {#"Lignes filtrées2"})

in
    #"TableAppend"

Sadly, I have this error when I try to run it : "Expression.Error: The columns parameter must be null, specify the number of columns, specify a list of column names, or specify a table type. Details: [List]"

So, is there anyone who have an idea of what I can do to integrate multiple sources and append them in a single request?

Answer 1

This is a very common problem:

1. Create a Parameter for Year and set it to 2021
2. Reference that parameter in your query for one year
3. Let PowerQuery create a Custom function from the result (right-click Create Function).
4. Enter all the years you are interested in a year column in another query
5. Add a new column by invoking your custom function and use the year column as parameter.
6. Expand the resulting tables and you're done

Read more about Using custom functions in the official documentation.

This is how your GetHolidays custom function should look like:

(Year as text) =>
let
    Source1 = Json.Document(
        Web.Contents(
            "https://canada-holidays.ca/api/v1/holidays?year=" & Year
        )
    ),
    #"Converti en table1" = Table.FromRecords({Source1}),
    #"ExpandList1" = Table.ExpandListColumn(#"Converti en table1", "holidays"),
    #"ExpandRecord1" = Table.ExpandRecordColumn(#"ExpandList1", "holidays", {
        "id", 
        "date", 
        "nameEn", 
        "nameFr", 
        "federal", 
        "observedDate"}, {
            "id", 
            "date", 
            "nameEn", 
            "nameFr", 
            "federal", 
            "observedDate"}),
    #"Type modifié1" = Table.TransformColumnTypes(#"ExpandRecord1",{
        {"id", Int64.Type}, 
        {"date", type date}, 
        {"nameEn", type text}, 
        {"nameFr", type text}, 
        {"federal", Int64.Type}, 
        {"observedDate", type date}}),
    #"Lignes filtrées1" = Table.SelectRows(#"Type modifié1", each (
        [nameEn] = "Boxing Day" or 
        [nameEn] = "Canada Day" or 
        [nameEn] = "Christmas Day" or 
        [nameEn] = "Day of Mourning for Queen Elizabeth II" or 
        [nameEn] = "Easter Monday" or 
        [nameEn] = "Good Friday" or 
        [nameEn] = "Labour Day" or 
        [nameEn] = "National Day for Truth and Reconciliation" or 
        [nameEn] = "New Year’s Day" or 
        [nameEn] = "Remembrance Day" or 
        [nameEn] = "Saint-Jean-Baptiste Day" or 
        [nameEn] = "Thanksgiving" or 
        [nameEn] = "Victoria Day"))
in
     #"Lignes filtrées1"

And this is your Holiday Calendar for all available years 2017-2026:

let
    Source = Table.FromList({2017 .. 2026}, Splitter.SplitByNothing(), {"Year"}),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Year", type text}}),
    #"Invoked Custom Function" = Table.AddColumn(
        #"Changed Type", "GetHolidays", each GetHolidays([Year])),
    #"Expanded GetHolidays" = Table.ExpandTableColumn(
        #"Invoked Custom Function", 
        "GetHolidays", 
        {"id", "date", "nameEn", "nameFr", "federal", "observedDate"}, 
        {"id", "date", "nameEn", "nameFr", "federal", "observedDate"}
    )
in
    #"Expanded GetHolidays"