Python & D3.js Dashboard Crossfilter Heat Map

Question

I've successfully replicated this interactive dashboard map example here.

I now want to input my own data. My python is more simple than the example's since all my data is already structured in the way I want it. Here's my python:

    # -*- coding: utf-8 -*-

import pandas as pd
from shapely.geometry import Point, shape

from flask import Flask
from flask import render_template
import json


data_path = './input/'

def get_location(longitude, latitude, ny_json):

    point = Point(longitude, latitude)

    for record in ny_json['features']:
        polygon = shape(record['geometry'])
        if polygon.contains(point):
            return record['properties']['Name']
    return 'other'

with open(data_path + '/geojson/ny_HA.json') as data_file:
    ny_json = json.load(data_file)

app = Flask(__name__)
    #route to html index
@app.route("/")
def index():
    return render_template("index.html")

    #route data
@app.route("/data")
def get_data():
    df = pd.read_csv(data_path + 'Mikes_NY_geocoded_tickets.csv', sep=';')

    df['location'] = df.apply(lambda row: get_location(row['longitude'], row['latitude'], ny_json), axis=1)

    cols_to_keep = ['timestamp', 'longitude', 'latitude', 'Violation', 'Type', 'DayOfWeek', 'location']

    #remove NaN values 
    df_clean = df[cols_to_keep].dropna()
    #return a json
    return df_clean.to_json(orient='records')


if __name__ == "__main__":
    app.run(host='0.0.0.0',port=5000,debug=True)

Here is my data geocoded_data.csv:

When I run my app python app.py, No errors and when I enter the localhost into my browser my dashboard appears, but everything is empty, my cross filters and map.

Here's what the browser is displaying:

Here are my errors in the browser console Any ideas would be much appreciated.

Answer 1

Now that you've shown your JS code (in the debugger screenshot above), it's pretty easy to guess what is going wrong.

You have

d["timestamp"] = dateFormat.parse(d["timestamp"]);
d["timestamp"].setMinutes(0); // Error: d["timestamp"] is null

Here's the documentation for d3.time.format.parse():

Parses the specified string, returning the corresponding date object. If the parsing fails, returns null. Unlike "natural language" date parsers (including JavaScript's built-in parse), this method is strict: if the specified string does not exactly match the associated format specifier, this method returns null. For example, if the associated format is the full ISO 8601 string "%Y-%m-%dT%H:%M:%SZ", then the string "2011-07-01T19:15:28Z" will be parsed correctly, but "2011-07-01T19:15:28", "2011-07-01 19:15:28" and "2011-07-01" will return null, despite being valid 8601 dates.

You haven't shown what the value of d["timestamp"] is prior to conversion, but chances are that there is a subtle difference between the expected format and the actual format.

In particular, if the screenshot of your data is correct, there are no seconds in your date format, but your d3.time.format specifier is expecting them?