Machine Learning In Node.js With TensorFlow.js

TF-JS

A simple example to implement an image classifier using Tensorflow.js

Reading about the library, I wanted to test it out with a simple task… 🧐

Before we dive into the code, let’s start with an overview of the different TensorFlow libraries.

Recent updates have extended the software to run in resource-constrained environments like mobile devices and web browsers.

The Node.js extension is an alpha release and still under active development.

Both Node.js extensions use native dependencies which will be compiled on demand.

One of the first challenges I encountered was that this does not work on Node.js.

The Node.js extension does not yet support HTTP requests to dynamically retrieve models. Instead, models must be manually loaded from the filesystem.

After reading the source code for the library, I managed to create a work-around…

Awesome, it works! But how where do the models files come from?

Models for TensorFlow.js consist of two file types, a model configuration file stored in JSON and model weights in a binary format. Model weights are often sharded into multiple files for better caching by browsers.

The HTTP retrieval code loads the model.json file from this location and then recursively fetches all referenced model weights shards. These files are in the format groupX-shard1of1.

Saving all model files to a filesystem can be achieved by retrieving the model configuration file, parsing out the referenced weight files and downloading each weight file manually.

Once this file has been downloaded locally, I can use the to parse all the weight file names.

Using the sed tool, I can prefix these names with the HTTP URL to generate URLs for each weight file.

Using the parallel and curl commands, I can then download all of these files to my local directory.

This does not work on Node.js due to the lack of a DOM.

Rather than trying to use an external package to simulate a DOM element in Node.js, I found it easier to construct the tf.Tensor3D manually.

pixels is a 2D array of type (Int32Array) which contains a sequential list of channel values for each pixel. numChannels is the number of channel values per pixel.

This will return a Uint8Array with four channel values ( RGBA) for each pixel ( width * height). The MobileNet model only uses the three colour channels ( RGB) for classification, ignoring the alpha channel. This code converts the four channel array into the correct three channel version.

Input values for images of different dimensions needs to be re-sized before classification. Additionally, pixels values from the JPEG decoder are in the range 0–255, rather than -1 to 1. These values also need converting prior to classification.

Developers can pass in Tensor3D inputs of type int32 and different dimensions to the classify method and it converts the input to the correct format prior to classification. Which means there's nothing to do... Super 🕺🕺🕺.

The tfjs-models/mobilenet library exposes a classify method on the MobileNet class to return the top X classes with highest probabilities from an image input.

predictions is an array of X classes and probabilities in the following format.

If everything worked, the following output should be printed to the console.

The image is correctly classified as containing a Panda with 99.93% probability! 🐼🐼🐼

TensorFlow.js brings the power of deep learning to JavaScript developers. Using pre-trained models with the TensorFlow.js library makes it simple to extend JavaScript applications with complex machine learning tasks with minimal effort and code.

Having been released as a browser-based library, TensorFlow.js has now been extended to work on Node.js, although not all of the tools and utilities support the new runtime. With a few days’ hacking, I was able to use the library with the MobileNet models for visual recognition on images from a local file.

Getting this working in the Node.js runtime means I now move on to my next idea… making this run inside a serverless function! Come back soon to read about my next adventure with TensorFlow.js. 👋

Related posts:

Continuous Integration (CI) is the practice of automatically running builds and tests to ensure the integrity of a codebase. What does this mean? Whenever someone proposes a change, CI quickly checks…