Why is Javascript causing my page to crash?

Question

so I'm using MVC 4 C#/Razor and I'm developing a page that uses SlickGrid to display grid data. Everything works fine, except when I try using it to display a large amount of data (something like 1 million rows).

When this happens, it appears to do just fine until it's just about finished. Right when it seems like it's going to be done with all dataloading, the web page crashes. I use getJSON to pull the data from a SQL database. I do it by column, and in batches of 300000 records. I have tried using Chrome memory profiling tools, and wasn't able to find anything useful. Below is some code snippets:

    function pullAllGridData(tableName, colArray)
    {
        for (var i = 0; i < colArray.length; i++)
        {
            fetchColumn(tableName, colArray[i], 0);
        }
    }


    function fetchColumn(tableName, fieldName, startAt)
    {
        $.getJSON('/WIMenu/GetTableData', { tableName: tableName, fieldName: fieldName }, function (data)
        {
            if (data.slice(-1) !== '~')
            {
                var startPass = populateSlickData(data, fieldName, startAt);
                colStatus[fieldName] = true;
                if (loadFirstBatch())
                { populateGrid(); }
                fetchColumn(tableName, fieldName, startPass);
            }
            else
            {
                data = data.slice(0, -1);
                populateSlickData(data, fieldName, startAt);
                colStatus[fieldName] = true;
                if (loadFirstBatch())
                { populateGrid(); }
            }
        });
    }

    function populateSlickData(input, fieldName, startAt)
    {
            var output = startAt;
            var valueArray = input.split('|');
            output += valueArray.length;
            if (!isInBlackList(fieldName, tableName))
            {
                var datatype = columns[getColumnIndex(fieldName)].datatype;
                var startIndex = startAt;
                var endIndex = startAt + valueArray.length;
                var counter = 0;
                alert(fieldName + ': startIndex: ' + startIndex + ' endIndex: ' + endIndex + ' count: ' + endIndex-startIndex);
                for (var x = startIndex; x < endIndex; x++)
                {
                    if (!slickdata[x])
                    { slickdata[x] = {}; }
                    if (valueArray[x - startAt] == 'null') { valueArray[x - startAt] = ''; }
                    if (datatype == 'System.DateTime')
                    {
                        if (valueArray[x-startAt] !== '')
                        {
                            var date = new Date(valueArray[x - startAt]);
                            valueArray[x - startAt] = (date.getMonth() + 1) + '-' + date.getDate() + '-' + date.getFullYear();
                        }
                    }
                    else if (datatype == 'System.Decimal' || datatype == 'System.Int32' || datatype == 'System.Int16' || datatype == 'System.Int64')
                    {
                        valueArray[x - startAt] = parseFloat(valueArray[x - startAt]);
                    }
                    slickdata[x][fieldName] = valueArray[x - startAt];
                    counter++;
                }
            }
            currentColumn = fieldName;

        filteredData = slickdata;
        return output;
    }

fetchColumn uses recursion to keep getting column data until all of it has been received. The populateGrid method simply syncs the SlickGrid object to the slickdata object. My goal here is to find out why the page is crashing and learn how it can be fixed.

Through using alerts, it seems that at some point, it gets stuck in the for loop in the populateSlickData method, and I cant figure out why. I've tried printing the for indexing data, but it all seems to be normal.

Answer 1

You can't pull a million rows of data into memory and expect any web page to do anything other than slow to a crawl, or indeed crash. This is what grid paging is for, coupled with on-demand ajax. Your grid should only pull the data needed to display the current page of data when the page is changed. You should not load everything ahead of time.

Here's an example on the SlickGrid github site: http://mleibman.github.io/SlickGrid/examples/example4-model.html

Here's more information: https://github.com/teleological/slickback/wiki/Pagination