Search code examples
javascriptlodashcrossfilter

Can you filter within a group with Crossfilter

I'd like to do some fast filtering in the browser and have been recommended crossfilter, but having looked at it I'm not entirely sure it suits my data. So, my data has just 3 variables:

{group: "A", value: 39.8, variable: "ABC"}
{group: "B", value: 26.8, variable: "ABC"}
{group: "C", value: 27.4, variable: "ABC"}
{group: "D", value: 26.9, variable: "ABC"}
{group: "E", value: 22.9, variable: "ABC"}
{group: "A", value: 48.9, variable: "ALL"}
{group: "B", value: 32.2, variable: "ALL"}
{group: "C", value: 16.2, variable: "ALL"}
{group: "D", value: 13.2, variable: "ALL"}
{group: "A", value: 42.3, variable: "ALL1"}
{group: "B", value: 50.1, variable: "ALL1"}
{group: "C", value: 19.3, variable: "ALL1"}
 etc

I'd like to be able to filter where the values limits vary for each group:

  (group = 'A' & 10 <= value <=20) or (group = 'B' & 15 <= value <=95) or
  (group = 'C' & 10 <= value <=20) or (group = 'D' & 25 <= value <=45) or
  (group = 'E' & 10 <= value <=20)

Is this possible without reorganising the data or are there better options?

So, this is what I've tried:

{group: "B", value: 26.8, variable: "ABC"},
{group: "C", value: 27.4, variable: "ABC"},
{group: "D", value: 26.9, variable: "ABC"},
{group: "E", value: 22.9, variable: "ABC"},
{group: "A", value: 48.9, variable: "ALL"},
{group: "B", value: 32.2, variable: "ALL"},
{group: "C", value: 16.2, variable: "ALL"},
{group: "D", value: 13.2, variable: "ALL"},
{group: "A", value: 42.3, variable: "ALL1"},
{group: "B", value: 50.1, variable: "ALL1"},
{group: "C", value: 19.3, variable: "ALL1"}]

var testcf = crossfilter(test)

const valueDimT = testcf.dimension(({group, value}) => ({group, value}));
valueDimT.filterFunction(({group, value}) => ((group = 'B' && value <= 30) || (group = 'C' && value <= 18)))
valueDimT.top(100)

Which gives the following result:

[{group: "C", value: 19.3, variable: "ALL1"}, 
{group: "D", value: 13.2, variable: "ALL"}, 
{group: "C", value: 16.2, variable: "ALL"}, 
{group: "E", value: 22.9, variable: "ABC"}, 
{group: "D", value: 26.9, variable: "ABC"}, 
{group: "C", value: 27.4, variable: "ABC"}, 
{group: "B", value: 26.8, variable: "ABC"}] (7) = $3

Which isn't what I would expect. Primarily I would expect only group values of B & C with values below the respective equalities given that's all that was in the filter:

[{group: "B", value: 26.8, variable: "ABC"},
{group: "C", value: 16.2, variable: "ALL"}]

Obviously I'm missing something - I'm just not sure what it is.

Solution

  • Using filterFunction

    You can filter using a function

    const valueDim = cf.dimension(({group, value}) => ([group, value]));
valueDim.filterFunction(([group, value]) => /* as above */)

    Note we are creating a composite key here. In an earlier draft I spelled the key {group, value} but that doesn't work because the dimension and group keys must be naturally ordered. An object won't work, because it coerces to the useless string [object Object] but an array will, because it coerces to a comma-delimited string with the values.

    Complete code:

    const valueDimT = testcf.dimension(({group, value}) => ([group, value]));
valueDimT.filterFunction(([group, value]) =>
  ((group == 'B' && value <= 30) || (group == 'C' && value <= 18)))
console.log(valueDimT.top(100))

    Fiddle demo.

    Creating a derived key

    It is probably more efficient to derive a key from the data:

    const condDim = cf.dimension(({group, value}) => /* as above */);
condDim.filterExact(true)

    JavaScript notes

    Careful with double comparisons:

    3 <= 2 <= 9

    evaluates to

    false <= 9

    or true in JavaScript, since false coerces to 0.

    You will need to do

    10 <= value && value <= 20

    Also be careful about = vs == - the former is assignment and the latter tests equality.