know what are new users added on a Firebase list, even after a restart

Question

As quoted in the documentation, I save users in a list. Users can log with both password and Facebook providers. (other providers may come later)

Then, I have a nodejs app that listens the user list and send me an email in case of new items.

This is easy with firebase, just add a child_added event listener. But as my app may restart for update/crash or any other reason, I do not want to get a email with every user at each time I restart the app. So I just save the latest userId that has been used to send an email.

var retrieveLatestAccountNotified = function () {
    ref.child("server-state").child("email-account").once("value", function (snapshot) {
        user = snapshot.val();
        console.log("Latest User Id : " + user);
        fetchUser(user);
        //fakeUser("");
    }, function (errorObject) {
        console.log("The read failed: " + errorObject.code);
    });
};


var fetchUser = function (latestUserId) {
    ref.child("users").orderByKey().startAt(latestUserId).on("child_added", function (snapshot, prevChildKey) {
        if (snapshot.key() !== latestUserId) {
            var newUser = snapshot.val();
            console.log(newUser);
            sendEmail.newAccountEmail(ses, newUser, snapshot.key(), function (err, data) {
                if (err)
                    throw err;
                console.log('Email sent:');
                console.log(data);
                ref.child("server-state").child("email-account").set(snapshot.key());
            });
        }

    }, function (errorObject) {
        console.log("The read failed: " + errorObject.code);
    });
};

The problem is that FB generates different userId type depending on the selected provider, so my mechanism will not work.

How to deal with that ?

• Add priority when the user is created => as many clients may add users, this mechanism has to be used in each client implementation
• Add a child (creation_date) in every user => adding extra info in user
• Add another list that saves which users have been notified => require to loop on both full list

Any educated proposition is welcome !

Answer 1

The common approach used for this is close to your #3:

Add another list that saves which users have been notified => require to loop on both full list

Typically you'd add a queue for sending the email notifications. This queue is purely for sending the notifications, so the server removes the item from it once it's done.

In its simplest form:

ref.child('emailNotificationQueue').on('child_added', function(snapshot) {
  sendEmailToAnthony(snapshot.val(), function() {
    snapshot.ref().remove();
  });
});

For better scalability and many other reasons, you might want to use firebase-queue. But it is a (beautifully) souped up version of such a loop.

You can take two approaches to the queue:

1. Your app writes the new user to /users and the notification to /emailNotificationQueue. This usually works, but has some nasty race/error conditions if a malicious client (or coding mistakes) writes one and not the other). While you can safeguard against these, there sometimes is a simpler approach.

2. Your app writes the new users to /newUserQueue. The server reads them from there, sends the email and adds them to /users.