Search code examples
node.jshashhashmaphashtablememoization

Memoizing traversed fs paths with Node.js


I have a tool that is doing concurrent searches through a filesystem for certain files. As the tool searches through the fs, it may find that it needs to search in directories not originally included in the search.

What I should do is memoize each directory that has already started to be searched.

I can't think of a better way to memoize file paths except to store them in a hash like this:

interface IMemoizationMap {
  [key: string]: boolean
}

so that might look like:

const hash = {
  '/Users/you/projects/x': true,
  '/Users/you/projects/x/lib': true,
  '/Users/you/projects/x/lib': true,
  ...
  ...
  '/Users/you/some-stuff/z': true
};

Then I do a quick lookup to see if I need to search a certain directory. What feels awkward about this solution is that the values in the hash could pretty much be anything - true, false, undefined.

Is this the best way to memoize traversed filepaths?

As an aside, is the performance of

key in hash

the same as

hash[key]

?

If that is the case, then there would be some worth to data stored as the value:

When a directory starts being searched I could make the value false, and then when the directory finishes searching I could flip the value to true. Then the value at least means something.


Solution

  • Go with Map:

    https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Map

    The Map object holds key-value pairs. Any value (both objects and primitive values) may be used as either a key or a value.

    Or Set:

    https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Set

    The Set object lets you store unique values of any type, whether primitive values or object references.

    I would choose Set, but i'm unaware of performance comparisons between both when testing if the value exists already on the collection.