Tag Archives: engineering tool

Self-contained Node.js Deployment

While setting up a Node.js environment on an individual developer’s machine can be done in a casual manner and oftentimes can be tailored to the developer’s own taste, deploying Node.js applications on shared or production servers requires a little more planning in advance.

To install Node.js on a server, a straight forward approach is to just follow some quick-start instructions from an official source. For instance, assuming latest v.4.x of Node.js is the target version and CentOS Linux is the OS on the target server, the installation can be as simple as follows:

Software version: Latest versus Same

However, the above installation option leaves the version of the installed Node.js out of your own control. Although the major release would stick to v.4, the latest update to Node available at the time of the command execution will be installed.

There are debates about always-getting-the-latest versus keeping-the-same-version when it comes to software installation. My take is that on individual developer’s machine, you’re at liberty to go for ‘latest’ or ‘same’ to suit your own need (for exploring experimental features versus getting ready for production support). But on servers for staging, QA, or production, I would stick to ‘same’.

Some advocates of ‘latest’ even for production servers argue that not doing so could compromise security on the servers. It’s a valid concern but stability is also a critical factor. My recommendation is to keep version on critical servers consistent while making version update for security a separate and independently duty, preferably handled by a separate operations staff.

Onto keeping a fixed Node.js version

As of this writing, the latest LTS (long-term-support) release of Node.js is v.4.4.7. The next LTS (v.6.x) is scheduled to be out in the next quarter of the year. There are a couple of options. Again, let’s assume we’re on CentOS, and that it’s CentOS 7 64-bit. There are a couple of options.

Option 1: Build from source

As a side note, if you’re on CentOS 6 or older, you’ll need to update gcc and Python.

Option 2: Use pre-built binary

Note that both the above two options install a system-wide Node.js (which comes with the default package manager NPM) accessible to all legitimate users on the server host.

Node process manager

Next, install a process manager to manage processes of the Node app, providing features such as auto-restart. Two of the most prominent ones are forever and pm2. Let’s go with the slightly more robust one, pm2. Check for the latest version from the pm2 website and specify it in the npm install command:

Deploying self-contained Node.js

Depending on specific deployment requirements, one might prefer having Node confined to a local file structure that belongs to a designated user on the server host. Contrary to having a system-wide Node.js, this approach would equip each of your Node projects with its own Node.js binary and modules.

Docker, as briefly touched on in a previous blog, would be a good tool in such use case, but one can also handle it without introducing an OS-level virtualization layer. Here’s how Node.js can be installed underneath a local Node.js project directory:

Next, create simple scripts to start/stop the local Node.js app (assuming main Node app is app.js):

Script: $PROJDIR/bin/njsenv.sh (sourced by start/stop scripts)

Script: $PROJDIR/bin/start.sh

Script: $PROJDIR/bin/stop.sh

It would make sense to organize such scripts in, say, a top-level bin/ subdirectory. Along with the typical file structure of your Node app such as controllers, routes, configurations, etc, your Node.js project directory might now look like the following:

Packaging/Bundling your Node.js app

Now that the key Node.js software modules are in place all within a local $PROJDIR subdirectory, next in line is to shift the focus to your own Node app and create some simple scripts for bundling the app.

This blog post is aimed to cover relatively simple deployment cases in which there isn’t need for environment-specific code build. Should such need arise, chances are that you might already be using a build automation tool such as gulp, which was heavily used by a Node app in a recent startup I cofounded. In addition, if the deployment requirements are complex enough, configuration management/automation tools like Puppet, SaltStack or Chef might also be used.

For simple Node.js deployment that the app modules can be pre-built prior to deployment, one can simply come up with simple scripts to pre-package the app in a tar ball, which then gets expanded in the target server environments.

To better manage files for the packaging/bundling task, it’s a good practice to maintain a list of files/directories to be included in a text file, say, include.files. For instance, if there is no need for environment-specific code build, package.json doesn’t need to be included when packaging in the QA/production environment. While at it, keep also a file, exclude.files that list all the files/directories to be excluded. For example:

Below is a simple shell script which does the packaging/bundling of a localized Node.js project:

Run bundling scripts from within package.json

An alternative to doing the packaging/bundling with external scripts is to make use of npm’s features. The popular Node package manager comes with file exclusion rules based on files listed in .npmignore and .gitignore. It also comes with scripting capability that to handle much of what’s just described. For example, one could define custom file inclusion variable within package.json and executable scripts to do the packaging/bundling using the variables in the form of $npm_package_{var} like the following:

Here comes another side note: In the dependencies section, a version with prefix ~ qualifies any version with patch-level update (e.g. ~1.2.3 allows any 1.2.x update), whereas prefix ^ qualifies minor-level update (e.g. ^1.2.3 allows any 1.x.y update).

To deploy the Node app on a server host, simply scp the bundled tar ball to the designated user on the host (e.g. scp $NAME-$VERSION.tgz njsapp@:package/) use a simple script similar to the following to extract the bundled tar ball on the host and start/stop the Node app:

Deployment requirements can be very different for individual engineering operations. All that has been suggested should be taken as simplified use cases. The main objective is to come up with a self-contained Node.js application so that the developers can autonomously package their code with version-consistent Node binary and dependencies. A big advantage of such approach is separation of concern, so that the OPS team does not need to worry about Node installation and versioning.

A Brief Encounter With Docker

Docker, an application-container distribution automation software using Linux-based virtualization, has gained a lot of momentum since it was released in 2013. I never had a chance to try it out but a current project has prompted me to bubble it up my ever-growing To-Do list. Below is a re-cap of my first two hours of experimenting with Docker.

First thing first, get a quick grasp of Docker’s basics. I was going to test it on a MacBook and decided to go for its beta version of Docker for Mac. It’s essentially a native app version of Docker Toolbox with a little trade-off of being limited to a single VM, which can be overcome if one uses it along side with Docker ToolBox. The key differences between the two apps are nicely illustrated at Docker’s website.

Downloading and installing Docker for Mac was straight forward. Below is some configuration info about the installed software:

Next, it’s almost illegal not to run something with a name by hello-world when installing a new software. While at it, test run a couple of less trivial apps to get a feel of running Docker-based apps, including an Nginx and a Ubuntu Bash shell.

While running hello-world or Ubuntu shell is a one-time deal (e.g. the Ubuntu shell once exit is gone), the -d (for detach) run command option for Nginx would leave the server running in the background. To identify all actively running Docker containers and stop them, below is one quick way:

It’s also almost illegal to let any hello-world apps sitting around forever, so it’s a perfect candidate for testing image removal. You’ll have to remove all associated containers before removing the image. Here’s one option:

Note that the above method only remove those containers with description matching the image name. In case an associated container lacking the matching name, you’ll need to remove it manually (docker rm ).

Adapted from Linux’s Cowsay game, Docker provides a Whalesay game and illustrates how to combine it with another Linux game Fortune to create a custom image. This requires composing the DockerFile with proper instructions to create the image as shown below:

Next, to manage your Docker images in the cloud, sign up for an account at Docker Hub. Similar to GitHub, Docker Hub allows you to maintain public image repos for free. To push Docker images to your Docker Hub account, you’ll need to name your images with namespace matching your user account’s. The easiest way would be to have the prefix of your image name match your account name.

For instance, to push the fortune-whalesay image to Docker Hub with account name leocc, rename it to leocc/fortune-whalesay:

Finally, it’s time to try actually dockerize an app of my own and push it to Docker Hub. A Java app of a simple NIO-based Reactor server is being used here:

The Dockerized Java app is now at Docker Hub. Now that it’s in the cloud, you may remove the the local image and associated containers as described earlier. When you want to download and run it later, simply issue the docker run command.

My brief experience of exploring Docker’s basics has been positive. If you’re familiar with Linux and GitHub, picking up the commands for various tasks in Docker comes natural. As to the native Docker for Mac app, even though it’s still in beta it executes every command reliably as advertised.