Create Our First Microservice

JavaScript Edition

Node.js is a JavaScript runtime built on Google Chrome's V8 JavaScript engine. In this step we're going to build a simple microservice using Node.js and the ioFog SDK.

Use Case

To keep things fairly simple, we'll want to have our microservice do something interesting, but not particularly complex. To get a feel for the primary functions of the SDK, we want a microservice that:

Uses dynamic configuration variables at runtime
Takes input from another microservice
Does some processing on it
Outputs new data from the results

Let's build a microservice that computes a real-time moving average from the input, which will send the result to any other microservices that might be listening. We'll also set up a dynamic configuration of the rolling window size. We can later change the configuration without needing to restart anything.

If we're in a hurry, we can skip ahead to the end.

Project Setup

Since we're going to be writing a new microservice, we'll need to create a project directory. Let's create it inside our tutorial's previous working directory that we should already be in.

Let's name our project "moving-average" and create our service's "main" entry point.

mkdir moving-average
cd moving-average
touch index.js

We then run npm init to set up our default Node.js package.json, providing the answers to all its questions and setting "main": "index.js"

npm init

Now we need to install the ioFog SDK for Node.js, which is published to NPM as @iofog/nodejs-sdk:

npm install --save @iofog/nodejs-sdk

The package.json file should look something like this now:

{
  "name": "moving-average",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@iofog/nodejs-sdk": "0.0.1"
  }
}

Using ioFog SDK

The ioFog Node.js SDK has a number of APIs, but in this tutorial we're most interested in these ones:

iofog.init()
iofog.getConfig()
iofog.wsControlConnection()
iofog.wsMessageConnection()
iofog.ioMessage()
iofog.wsSendMessage()

First, include the ioFog SDK in our index.js.

const iofog = require('@iofog/nodejs-sdk');

iofog.init()

We now have to register a callback for ioFog once the ioFog SDK has finished initializing. It accepts a number of arguments, but we'll most likely want to pass these defaults. Here we register main() function as the init callback.

iofog.init('iofog', 54321, null, main);
});

For the curious, the first argument is the host name of the Agent's Local API, the second is the port number of the SDK, and the third can be the container's ID, though it is not required.

iofog.getConfig()

Asynchronously fetch the microservice's current configuration (config). When it starts, the configuration of the microservice is only read once.

function updateConfig() {
  iofog.getConfig({
    onNewConfig: newConfig => {
      config = newConfig;
    },
    onBadRequest: err => console.error('updateConfig failed: ', err),
    onError: err => console.error('updateConfig failed: ', err)
  });
}

iofog.wsControlConnection()

Note that when a configuration of a microservice changes, the Controller will send a message to the involved microservice.

Therefore we have to connect the ioFog control channel via WebSocket, which is used to receive notifications from the Controller that our microservice's config has changed.

Because a config can be any arbitrary JSON, including very large files, the change notifications themselves do not actually include the config. So if we would like to update our local cache of the config, we have to follow up a change notification with a call to iofog.getConfig().

iofog.wsControlConnection({
  onNewConfigSignal: () => updateConfig(),
  onError: err => console.error('Error with Control Connection: ', err)
});

iofog.wsMessageConnection()

Next, we have to connect to the ioFog message channel via WebSocket. This is where we'll receive any messages routed to this microservice from another.

Under the hood, communication is brokered by our Connector and messages are routed according to that microservice's route settings on the Controller.

iofog.wsMessageConnection(onMessageConnectionOpen, {
  onMessages: messages => {
    // Do something with messages...
  },
  onMessageReceipt: (messageId, timestamp) => {
    console.log('message receipt: ', {
      messageId,
      timestamp
    });
  },
  onError: err => console.error('Message WebSocket error: ', err)
});

iofog.ioMessage() / iofog.wsSendMessage()

Now that we can read control signals and message, we also need to send messages out with the actual moving average. We create and send ioMessages in JSON, which is the Node.js serialization format used for intercommunication between microservices.

When our code wants to publish a message to any other microservice, these are what we'll be sending.

There are a number of optional fields, but the most common are: contentdata, infotype, and infoformat. The contentdata field is the actual data payload we want to send, which needs to be provided as a string.

const output = iofog.ioMessage({
  contentdata: Buffer.from(JSON.stringify(result)).toString(),
  infotype: 'application/json',
  infoformat: 'text/utf-8'
});

iofog.wsSendMessage(output);

Putting The Moving Average Together

We're ready to start writing some code! First, fire up our favorite editor and open (or create) the index.js file we set as our package.json "main". This is where we'll place all of our code.

Before we begin, let's review our goals for our moving average microservice:

Window size should be configurable
Take input from another microservice
Compute a moving average on that input
Output new data from the results

We'll want to have our microservice expect a custom config with a maxWindowSize field telling us what the max size of our rolling window should be.

To compute our real-time moving average, we can first create some utilities to compute an average from any array of numbers:

const sum = values => values.reduce((a, b) => a + b, 0);
const average = values => sum(values) / (values.length || 1);

average([2, 6, 14, 53, 87]); // returns 32.4

In order to do a rolling window, we'll store incoming values in an array up until the point where a max window size is reached. After which we'll discard the oldest value, computing a new average each time.

function getMovingAverage(arr, newValue) {
  // Evict the oldest values once we've reached our max window size.
  while (arr.length >= config.maxWindowSize) {
    // <------- config
    arr.shift();
  }
  arr.push(newValue);
  return average(arr);
}

The Sensors microservice produces objects that look like this:

{
  "time": 1540855847710,
  "speed": 41.71445712,
  "acceleration": "0.52431",
  "rpm": "2078.3"
}

What we'll do is produce an average for speed, acceleration, and rpm.

onMessages: messages => {
  if (messages) {
    for (const msg of messages) {
      const input = JSON.parse(msg.contentdata.toString());

      // Produce moving averages for all the sensor values
      const result = {
        isAverage: true,
        time: input.time, // same time as
        speed: getMovingAverage(prevSpeeds, parseFloat(input.speed)),
        acceleration: getMovingAverage(
          prevAccelerations,
          parseFloat(input.acceleration)
        ),
        rpm: getMovingAverage(prevRpms, parseFloat(input.rpm))
      };

      const output = iofog.ioMessage({
        contentdata: Buffer.from(JSON.stringify(result)).toString(),
        infotype: 'application/json',
        infoformat: 'text/utf-8'
      });

      iofog.wsSendMessage(output);
    }
  }
};

The Final Moving Average Code

We now have everything we need to complete our microservice!

const iofog = require('@iofog/nodejs-sdk');

// Used as our in-memory cache of our configuration
// that will be provided by the Controller
let config = null;

function updateConfig() {
  iofog.getConfig({
    onNewConfig: newConfig => {
      config = newConfig;
    },
    onBadRequest: err => console.error('updateConfig failed: ', err),
    onError: err => console.error('updateConfig failed: ', err)
  });
}

const sum = values => values.reduce((a, b) => a + b, 0);
const average = values => sum(values) / (values.length || 1);

function getMovingAverage(arr, newValue) {
  // Evict the oldest values once we've reached our max window size.
  while (arr.length >= config.maxWindowSize) {
    // <------- config
    arr.shift();
  }
  arr.push(newValue);
  return average(arr);
}

// This is basically our "entry point", provided to iofog.init() below
function main() {
  updateConfig();

  iofog.wsControlConnection({
    onNewConfigSignal: () => updateConfig(),
    onError: err => console.error('Error with Control Connection: ', err)
  });

  const onMessageConnectionOpen = () => {
    console.log('Listening for incoming messages');
  };

  // Cache for our previous values received so we can compute our average
  const prevSpeeds = [];
  const prevAccelerations = [];
  const prevRpms = [];

  iofog.wsMessageConnection(onMessageConnectionOpen, {
    onMessages: messages => {
      if (messages) {
        for (const msg of messages) {
          const input = JSON.parse(msg.contentdata.toString());

          // Produce moving averages for all the sensor values
          const result = {
            isAverage: true,
            time: input.time, // same time as
            speed: getMovingAverage(prevSpeeds, parseFloat(input.speed)),
            acceleration: getMovingAverage(
              prevAccelerations,
              parseFloat(input.acceleration)
            ),
            rpm: getMovingAverage(prevRpms, parseFloat(input.rpm))
          };

          const output = iofog.ioMessage({
            contentdata: Buffer.from(JSON.stringify(result)).toString(),
            infotype: 'application/json',
            infoformat: 'text/utf-8'
          });

          iofog.wsSendMessage(output);
        }
      }
    },
    onMessageReceipt: (messageId, timestamp) => {
      console.log('message receipt: ', {
        messageId,
        timestamp
      });
    },
    onError: err => console.error('Message WebSocket error: ', err)
  });
}

iofog.init('iofog', 54321, null, main);

Create Dockerfile

We now to need to package up our code as a Docker image, so that we can deploy it in the next step. Docker images are created from instructions written in a Dockerfile.

Like all build scripts, Dockerfiles can become a bit complex for advanced applications, but fortunately, ours is fairly simple:

FROM node:8
COPY . /moving-average
WORKDIR /moving-average
RUN npm install --only=production
CMD node .

Build Our Docker Image

With our Dockerfile setup, we can go ahead and build our image:

docker build --tag iofog-tutorial/moving-average:v1 .

We'll wait a few minutes while it downloads a default Node.js environment we're using as a base.

Let's double check the images were successfully created. The image name and tag are important in the next step of the tutorial, where we are going to deploy the moving average service.

docker image ls --filter 'reference=*/moving-average'

Deploy Our Microservice

We now want to see this code in action, so let's go ahead and learn how to deploy this microservice to our ioFog tutorial environment.

Continue To Next Step: Deploy Our Microservice.