Context in Highland.js

Question

I love Highland.js and the style of reactive programming in general. I'm struggling with the loss of context and I'm trying to determine how to elegantly handle context in a model where the goal is to abandon state.

As an example, I have an array of my accounts in Amazon Web Services.

var accounts = [{accessId:"12345","secretKey":"abc123","account":"foo"},
                {accessId:"34512","secretKey":"def456","account":"bar"}];

My goal is to basically create a spreadsheet of all of my EC2 instances running in a region. Something like this.

Account   | Instance Size
--------- | -------------
foo       | m3.xlarge
foo       | c3.medium
bar       | t2.small

The general workflow would be

1. Go through each account
2. Call ec2DescribeInstaces
3. Somehow map each ec2DescribeInstances call back to the account name for final output

In normal JavaScript, we would be doing looping here so when we make each call to ec2DescribeInstances, a context would exist

for ( account in accounts ) {
  var instances = ec2DescribeInstaces(account);
  for ( instance in instances ) {
    results.push({account:account.name, instanceSize: instance.size});
  }
}

From what I understand in reactive programming, I would do something like this

_(accounts)
 .map(ec2DescribeInstaces)
 .parallel(2)
 .each(function(result) {
   results.push(result);
 });

Any guidance??? So at the end of this chain, I have the instances from Amazon. But I'm not sure how to tie those back to the accounts to get the name. I know I can hack around this to get the value but I'm looking for best practices and something elegant.

So @Bergi, like below??? This would essential return a "context" object with both the data that I need in addition to the data that came back from Amazon. My only concern with this is that if I'm passing a context about throughout the chain then we will be doing a lot of data plucking, stuffing, and wrapping for calls.

ec2DescribeInstances = _.wrapCallback(function(accountData, callback) {
  // we remove the extraneous account name using a pick
  var ec2 = new AWS.EC2(lodash.pick(accountData,'accessKeyId','secretAccessKey'));
  ec2.describeInstances({ Filters: [{Name:'instance-state-name', Values:['running']}] }, function(err,data) {
    if ( err ) return callback(err);
    // here I create an object that wraps the AWS response
    callback(null, {"account":accountData.alias, "data": data})
  });
});

Answer 1

You can use _.zip(), which takes two streams and returns a stream of pairs. So, using your stream that generated ec2Instances as an example:

var ec2InstanceStream = _(accounts)
    .map(ec2DescribeInstaces)
    .parallel(2);

_(acounts).zip(ec2InstanceStream);
    .each(function(result) {
        //Each result: [{ /* original account */ }, { /* ec2 instance */ }]
        results.push(result);
    });

So now you have everything together. You can make it nicer of course. For instance, if you want to merge each pair into a single object, you could add a .map(_.extend) before the .each() step to merge the objects together.