Search code examples
xmlcsvxpathxquerybasex

how to use expath and Xquery to fetch a CSV file?


How can this example of expath be adapted to grab and store a CSV file directly from a website?

xquery version "3.1";

import module namespace hc = "http://expath.org/ns/http-client";

let $url := "https://www.kingjamesbibleonline.org/Genesis-Chapter-1_Original-1611-KJV/"
let $request := 
    <hc:request href="{$url}" method="GET">
        <hc:header name="Connection" value="close"/>    
    </hc:request>
return
    hc:send-request($request)

Browsing to:

http://www.bccdc.ca/health-info/diseases-conditions/covid-19/data

the specific CSV file is:

http://www.bccdc.ca/Health-Info-Site/Documents/BCCDC_COVID19_Dashboard_Case_Details.csv

Of course, there are a multitude of ways to grab that file. But, how would it be done with xquery as above?

Obviously, changing the URL is the starting point, which, indeed, results in a large CSV file printed to the console.


Solution

  • As @MartinHonnen already pointed out, BaseX has a CSV module.

    Here is how to use it.

    csv file addressbook.csv

    FName, LName, Address, City
    John, Doe, 3851 SW 52nd St., Miami
    Mary, Dowson, 770 SE 21nd St., Orlando
    

    XQuery

    let $text := file:read-text("c:\Users\Yitzhak\Downloads\addressbook.csv")
    return csv:parse($text, map { 'header': true() })
    

    Output

    <csv>
      <record>
        <FName>John</FName>
        <LName> Doe</LName>
        <Address> 3851 SW 52nd St.</Address>
        <City> Miami</City>
      </record>
      <record>
        <FName>Mary</FName>
        <LName> Dowson</LName>
        <Address> 770 SE 21nd St.</Address>
        <City> Orlando</City>
      </record>
    </csv>