Search code examples
javascriptarraysperformanceoptimizationapi-design

API/Application Design: Replace with very large array with a new array or modify the existing array

I have a very large array of objects (nested objects).

var array = [
    { a: "a", b: { .. }, c:"c", ... },
    {...}
]

After some API call, I get a brand new array with 1 modified element and I know exactly which element is modified.

  1. Is it a good idea (in terms of memory usage and performance) to assign array with the new value or replace only the modified object.

  2. Do I need to modify the API to only send the modified object and update the array accordingly?

The API is developed by another team and it transfers a huge amount of data on each request. I need a solid technical answer to convince them to change the API to send only the required data and do modification at the client side.

Solution

  • If by

    "I know exactly which element is modified"

    you mean you know the exact position of the modified element, then replacing it is an O(1) operation:

    array[positionOfModified] = modified;

    Otherwise, you will have to find the element, which is usually an O(N) operation unless you do something like a binary search if the array is sorted (O(logN)).

    Therefore, in terms of speed, it could potentially be slower to replace the modified object then to just replace array references:

    array = newArray;

    However, the space (memory) improvement would likely be much larger than the possible speed regression.

    Returning only the modified element will reduce your bandwidth since you'll be sending a single object instead of a large array. If this request happens frequently (many users requesting many times, possibly simultaneously), by returning the whole array every time, you are risking congesting your network.

    The application memory usage will also be improved because you'll be overwriting a single object instead of an array, thus the garbage collector will only have to worry about cleaning up the modified object, not the entire previous array. Replacing references of large arrays, especially if this replacement is done often (maybe faster than the GC does its cleanup cycles), could blow up your memory fairly quickly.

    Ideally, what you could do is send the modified object and its position in the array back, something like:

    {
  element: { ... }
  position: ...
}

    this will allow you to use small memory/bandwidth while keeping the update process a constant operation.

    array[response.position] = response.element;