I have a small data conversion application that converts xml into a json then stores it in a mySql DB, using node, express & node-mysql
it works very well until I do large batches, this has nothing to do with the module or mySql but it seems that there is some trickery going on in node.
To format the data for an insert I need to have the data in a basic Array so I do something like this.
var keys = ['first_name', 'last_name', 'mi'], values, que, i, q;
values = []
que = []
i = 0
// data is a json array with corresponding keys
while(i < data.length){
values.length = 0
q = 0
for(key in data[i]){
values.push(data[i][keys[q]]);
q += 1
}
que.push(values)
verifyInsert(i)
i += 1
}
When i do this with one entry the app works wonderful, but when i do multiple i get the last entry many times.
There is nowhere that i overwrite que but for some reason after each iteration the all the values in que change. In normal javascript I use this method all the time to construct an array with reusable variable, then I store the instance in another location. Destroy the values in the reusable variable values.length = 0.
Does this method not work in node? because it seems that im storing the variable itself not the instance of the variable.
Seems like that's your problem, that its storing the variable itself.
To overcome this, you should do
values = [];
in your loop. That will create a new array for each iteration