Search code examples
javascriptpythoncsvaws-lambdahttprequest

Simplest ways of getting CSV headers from URL

I've built a web-app that where users can upload contacts using CSV files.

Once the CSV file is uploaded, I'd like to map the header fields to the app's fields.

The issue I'm facing is figuring out an efficient and simple way to return a list of CSV headers from a CSV link.

Ideally, I'd like the list to return via an HTTP request; or alternatively, using JavaScript, so I can pass it back to the app.

For example, here is a CSV file containing hurricane counts: https://people.sc.fsu.edu/~jburkardt/data/csv/hurricanes.csv

The headers are:

Month, "Average", "2005", "2006", "2007", "2008", "2009", "2010", "2011", "2012", "2013", "2014", "2015"

My idea is to run a Python script in AWS Lambda that gets the CSV headers, then sends them to the app using an HTTP request.

Is there another way to get this done client-side or without having to setup a backend infrustructure like AWS Lambda?

Solution

  • There's an option to stream from requests and you can process CSV per line. See: Body Content Workflow

    Example:

    import requests

url = "https://raw.githubusercontent.com/codeforamerica/ohana-api/master/data/sample-csv/addresses.csv"
r = requests.get(url, stream=True)

for line in r.iter_lines():
    header = line.decode("utf-8").split(",")
    print(header)
    break

    This will give you only the header. I've used an example of raw CSV file from github: sample-csv