Search code examples
javascriptarraysjsondata-cleaning

Cleaning out bad JSON data from object?


I am retrieving building geolocation data from a service, however in some cases some of the data will have missing properties.

Good data

[{
    "ModifiedOn": "2015-04-29 11:17:28.0",
    "BuildingName": "Wintefell",
    "Latitude": "52.900619",
    "Longitude": "-1.434602",
    "Region": "The North",
}, {
    "ModifiedOn": "2015-04-29 11:17:28.0",
    "BuildingName": "Water Gardens",
    "Latitude": "51.818051",
    "Longitude": "-.354871",
    "Region": "Dorne"
}]

Bad data

Missing Region or Building Name

{
    "ModifiedOn": "2015-04-29 11:17:28.0",
    "BuildingName": "Kings Landing",
    "Latitude": "23.818051",
    "Longitude": "-.154871",
}

After making my AJAX request I store the JSON response in an object called _regionAndBuildings

I want to clean out any bad data from it, so I tried the following code

console.log("Starting size of building data : " + _regionAndBuildings.length);
//clean json by setting object to undefined
for (var i = 0; i < _regionAndBuildings.length; i++) {
    if (_regionAndBuildings[i].BuildingName === undefined && _regionAndBuildings[i].Region === undefined) {
        console.log("Deleting");
        delete _regionAndBuildings[i];
    }
}
console.log("Ending size of building data after cleaning : " + _regionAndBuildings.length);

Output

Starting size : 209

Delete x 17

Ending size : 209

Why is this code not removing the bad elements in my JSON object?


Solution

  • There are two problems in your code.

    1. As per the question

      Missing Region or Building Name

      You want to remove the elements of either Region or Building Name is missing. But the condition doesn't reflect that

      if (_regionAndBuildings[i].BuildingName === undefined && 
          _regionAndBuildings[i].Region === undefined)
      

      Since you have used && operator, only when both the expressions are satisfied, the code inside if condition will be executed. So, you need to change it with logical OR operator ||, like this

      if (_regionAndBuildings[i].BuildingName === undefined ||
          _regionAndBuildings[i].Region === undefined)
      
    2. When you use delete operator on an array, the element in the array will be deleted, but the index will point to nothing. It means the length of the array will not change. Read more about it in this section

      Instead, you can use Array.prototype.splice, like this

      _regionAndBuildings.splice(i, 1);
      

    But, I prefer reconstructing the array without the elements you don't want, with Array.prototype.filter, like this

    var filteredBuildings = _regionAndBuildings.filter(function (currentObject) {
        return currentObject.hasOwnProperty("BuildingName") &&
            currentObject.hasOwnProperty("Region");
    });
    

    Now, the function passed will be run with all the items in the array and only if the function returns true, the object will be included in the filtered result. So, if the object has both BuildingName and Region properties, only then it will be included in the result.


    Note: I prefer using Object.hasOwnProperty to check if the property exists in the object, rather than comparing the value against undefined, because if the value of the property is actually undefined, then we cannot differentiate between a non-existing property and a property with value undefined.