How can I create a "flattened" group from an array of arrays?

Question

I have a crossfilter built from a list of posts made by a user, and I want to use the data to drive a word cloud-style visualisation.

My records are an array of objects like this:

[
    {
      "date": Thu May 24 2018 11:29:27 GMT+0100 (GMT Daylight Time),
      "text": "Lorem ipsum dolor sit amet"
    },
    ...etc
]

I've set up some dimensions on the date field to filter my records, and I'd like the contents of the word cloud to update to reflect the filtered data. I'm not sure how to set up a group that is a flat array of every word in the filtered post.

Setting up a dimension like this

var textDimension = ndx.dimension(p => p.split(' '))

will give

[
    ['Lorem', 'ipsum', 'dolor',  'sit', 'amet'],
    ...etc
]

which is fine but there doesn't seem to be a way to do textDimension.group(/* do something...*/).reduceSum(r => 1) that treats the dimension as

[
    'Lorem',
    'ipsum',
    'dolor',
    'sit',
    'amet'
]

which after the reduceSum would produce something like

[
    {"key": "Lorem", "value": 1},
    {"key": "ipsum", "value": 1},
    {"key": "dolor", "value": 1},
    {"key": "sit", "value": 1},
    {"key": "amet", "value": 1},
]

instead of

[
    {"key": ['Lorem', 'ipsum', 'dolor',  'sit', 'amet'], "value": 1},
]

which is how it currently does with do logic in group(). I could do what I want with

var filteredPosts = ndx.allFiltered()
var wordsByPost = filteredPosts.map(p => p.split(' ')
var allWords = [].concat.apply([], wordsByPost)

and then count up the instances of each word in allWords (potentially by setting up a new crossfilter) but it feels like the wrong way to do it.

Is there a way to do this using the crossfilter I've already got set up or do I admit defeat and use the basic JS solution?

Answer 1

If you are using Crossfilter 1.4+, then you just need to tell it that the dimension is an array dimension like so:

var textDimension = ndx.dimension(p => p.split(' '), true)

Documentation: https://github.com/crossfilter/crossfilter/wiki/API-Reference#dimension_with_arrays