Push Notification Made Easy with MongoDB and GlingJS

2018-04-02

GlingJS is a small component that lets you pump data events into the browser from MongoDB.

Imagine telling a customer lingering on a product catalog page “Hey! Someone just bought this very product!” or the animal shelter page popping up a picture every time a kitten is saved. Let your imagination run wild, you do you.

Cat Meme

The project was inspired by the recent introduction of MongoDB Change streams. Change streams are an efficient way to track changes occurring at the data level. It notifies you in real time each time a document is written or changed. You just need to specify which collection and under what data change conditions you want to get notified.

Ok, sounds lovely, but why GlingJS? Why another library on top of the node driver syntax?

There’s a huge number of MEAN stack and node developers.
They mostly build web applications.
Real-time notification with WebSocket is better than polling or waiting for page transition.
MongoDB is powering a staggering number of websites.

With GlingJS, you get real-time messages in the browser, with very little code.

Theory & Practice

First, you’d want to grab the package from NPM

npm install glingjs --save

Server Side

On the Node side, GlingJS consists largely of 2 working components: the ClientManager and Gling itself. In addition, WebSocketServerManager glues the WebSocket transport to the ClientManager

Client Manager

The client manager is the component that manages the WebSockets. It handles browser connection requests and topics. A topic is a subject that the browser might be interested in. Things like “when a customer registers” could be broadcast to the “newUser” topic. These are made up - they depend on your requirements so there’s no built-in set of hard-coded topics.

The client manager exposes a callback function broadcast(topic, message). This hook is what Gling uses to emit the database event upstream to the browser.

Your app would need one instance of clientManager, created thusly:

const clientManager = new ClientManager(config);

Web Socket Server Manager

The WebSocket Server manager is really just glue: WebSocketServerManager is instantiated to latch on to your Node http server, and attach WebSocket functionality to the Client Manager.

var webSocketServerManager = new WebSocketServerManager(httpServer, clientManager);

If your node app already has a constructed http server, you’d hand it to the component. If you are running a standalone Node instance(s) just for Gling, then you would need to set up an http server before reaching this point.

Gling

Gling is the component that deals with MongoDB change streams. It is also configuration based, so getting an instance of Gling is straightforward:

var gling = new Gling(config);

Gling exposes a function start(hook) which takes a callback matching the signature of ClientManager.broadcast.

Starting Gling looks something like:

// passing the ClientManager instance broadcast hook to Gling
gling.start(clientManager.broadcast)
    .then(() => console.log('started gling..'))
    .catch(reason => {
        console.error(reason);
    });

Gling handles all the business of subscribing to the MongoDB using the driver API, so you don’t have to.

Client Side

Browser

GlingJS uses WebSocket to perform the server side work.

Most browsers nowadays support the WebSocket standard, so no special library required.

The browser needs to open a WebSocket connection to the server, and provide the topic it is interested in.

<html>
    <head>
        <script type="text/javascript">
        (function () {
            // subscribing to 'meme-cat' topic, on host 'myservername'
            var ws = new WebSocket("ws://myservername:80/gling", "meme-cat");

            ws.onmessage = function (evt) {
                const data = JSON.parse(evt.data);
                //TODO: show data using your UI
            };

            window.onbeforeunload = function (event) {
                ws.close();
            };
        })();
        </script>

In the example above the web page subscribes to the topic meme-cat coming from the server myservername. Another page may subscribe to a different topic.

You might also notice window.onbeforeunload browser event, which disconnects the socket when a browser closes. It is good citizenship which helps the server clean up the server side resources and would help keep overhead low.

We mentioned topic a few times and eluded to a configuration. Let’s dig deeper!

Configuration

Listeners

For notifications to be useful, a client (web browser) would expect a few things:

That it gets messages it is interested in
That it doesn’t get messages it is not interested in
That the messages it receives have the field(s) it expects

The first 2 requirements really talk about the topic. The topic is pre-defined, and acts as a filter here: the browser says it wants cat memes, we’re going to send cat memes, but no dog memes.

The client also expects that certain fields are present. As a good convention, we’ll aim to have a single topic imply a set schema: all messages for a topic should have the same set of fields. Gling doesn’t enforce a schema, but by nature, it subscribes a topic to the same underlying documents, so the uniformity of those helps the browser get uniformly shaped messaged. Bottom line: your browser should expect that a single topic will receive the same type of message. It is possible to manufacture 2 different events and pump them under the same topic, but that can get really silly, and topics are free, so just come up with a new topic name.

Here is an example configuration:

var ChangeType = require('../src/changeType');

const Config = {
    connection: 'mongodb://localhost:27017/gling?replSet=r1',
    allowedOrigins: ['https://example.com/gling'],
    listeners: [
        {
            collection: 'memes',
            when: [ChangeType.create],
            filter: { about: 'cat' },
            fields: ['url','caption'],
            topic: 'meme-cat'
        }, 
        //...
    ]
}

module.exports = Config;

Topics get created inside the listeners array. The fields control the delivery of the notifications:

Field	Description
`collection`	The MongoDB collection name
`when`	Which type of changes we care about. This can be any of ‘create’, ‘update’, ‘remove’
`filter`	Is a match condition. Only documents that match this condition would be returned.
`fields`	Which fields from the document would be returned
`topic`	Topic name

A multitude of listeners can be defined and supported by a single Node server, as long as it can handle the volume of notifications from Mongo and number of simultaneous browsers connected.

Filters can be compound expressions and include any operators that the aggregation framework supports. They are static though. Since change stream subscription is done once, you can’t keep changing the value in a match expression. That is to say, if you pass in the value {created: {$gt: Date()}}, the date compared will be the time the Node server started the process, not evaluated each time a document changes.

Field trimming is optional but allows you to shrink down the amount of data you send to the browser. This can help performance and also security. Given a Mongo document

{_id:12, name: 'bob', city:'La Paz', password: 'password123'}

You can configure fields to include only ['city','name'] and suppress data that should not be sent to the browser.

WebSocket Safety

To help protect browsers against scripting attacks, it’s best if you explicitly allow your domain(s) only. Pages on different domains would be rejected.

Setting allowedOrigins array is there for the ClientManager and your http server setup. ClientManager exposes a convenience method isOriginAllowed(thisOrigin, listOfAllowedOrigins) which does the math to figure if the current request origin should be allowed or not.

WebSocketServerManager takes care to reject origins not specified in the configuration.

An asterisk ‘*’ in the list of allowed origins will allow any origin! Use it in local development only, never in production!

Mongo Connection

The MongoDB server connection is specified using the connection field. Any legal MongoDB URI string would work. But since MongoDB Change Streams are built on the Oplog, the MongoDB server should be part of a replica set (even a replica-set of one server would do). For that reason, you should include the URL parameter replSet= to set the replica set name of your MongoDB cluster.

Conclusion

GlingJS is a simple and easy way to push notifications from MongoDB document change events directly into a browser using WebSockets. A demo project can be found here. The first part sets up an http server. The code to actually set up Gling and the ClientManager is minimal and simple. Check it out, see what you think!

+N Consulting, Inc.