Search code examples
d3.jsdata-visualizationlarge-data

d.3 dealing with half million entries

I am trying to load 1/2 million data point to the "simple graph" example. I guess there may be a limit for SVG rendering, because the line is not showing up. Can you advice me on how to deal with this "big" graph?

Thanks

// Here the code 

<!DOCTYPE html>
<meta charset="utf-8">

<style>

path{stroke: steelblue;
    stroke-width: 4
    ;
    fill: none;}

.axis path,
.axis line {fill: none;
            stroke: grey;
            stroke-width: 2.5;
            shape-rendering: crispEdges;}

</style>

<h1>FDD Visualization</h1>
<h2>A simple Graph with visual aids using d3</h2>

<body>
<script type="text/javascript" src="d3/d3.v3.js"></script>
<script type="text/javascript"></script>

  <script>

var margin = {top: 50, right: 20, bottom: 50, left: 50},
    width = 1020 - margin.left - margin.right,
    height = 580 - margin.top - margin.bottom;

var parseDate = d3.time.format("%Y-%m-%d").parse;

var x = d3.time.scale().range([0, width]);
var y = d3.scale.linear().range([height, 0]);

var xAxis = d3.svg.axis().scale(x)
    .orient("bottom").ticks(5);

var yAxis = d3.svg.axis().scale(y)
    .orient("left").ticks(5);

var valueline = d3.svg.line()
    .x(function(d) { return x(d.timestamp); })
    .y(function(d) { return y(d.temperature); });

var svg = d3.select("body")
    .append("svg")
        .attr("width", width + margin.left + margin.right)
        .attr("height", height + margin.top + margin.bottom)
    .append("g")
        .attr("transform", "translate(" + margin.left + "," + margin.top +")")
        .attr("fill",function(d){return"rgb(0,100,10)";} );

// Get the data
d3.csv("data/data_2.csv", function(error, data) 
{data.forEach(function(d) 
    {d.timestamp = parseDate(d.timestamp);
     d.temperature = +d.temperature;});

    // Scale the range of the data
    x.domain(d3.extent(data, function(d) { return d.timestamp; }));
    y.domain([0, d3.max(data, function(d) { return d.temperature; })]);

    svg.append("path")      // Add the valueline path.
        .attr("d", valueline(data));

    svg.append("g")         // Add the X Axis
        .attr("class", "x axis")
        .attr("transform", "translate(0," + height + ")")
        .call(xAxis);

    svg.append("g")         // Add the Y Axis
        .attr("class", "y axis")
        .call(yAxis);

});

</script>
</body>

// Here the data (only first rows...it amounts for approx. 500,000 rows)

timestamp,temperature
01/01/1900,23
02/01/1900,13
03/01/1900,13
04/01/1900,15
05/01/1900,12
06/01/1900,24
07/01/1900,12
08/01/1900,19
09/01/1900,12
10/01/1900,16
11/01/1900,16
12/01/1900,19
13/01/1900,24
14/01/1900,12
15/01/1900,20
16/01/1900,20
etc...
Solution

  • Nothing displays because your dates aren't being parsed correctly.

    Your dates looks like this:

    16/01/1900

    but parseDate is using the format specifier:

    "%Y-%m-%d"

    Fixing this issue just requires a one line change: 

    var parseDate = d3.time.format("%d/%m/%Y").parse;

    A suggestion: use the dev console to debug things! Trying to run your script shows an error:

    Error: Problem parsing d="MNaN,20.000000000000057LNaN,220LNaN,220LNaN, ... "

    which doesn't look like is has anything to do with trying to display too many values. Inspecting the data array, all the temperature values are undefined after parseDate is called on them.

    Finally, I think Pablo's advice about not trying to show a line graph w/ 500,000 points is solid. Our screens and eyes don't have a high enough resolution to make that a useful display