A Simple Background Processing Framework

While there are true threading libraries for Node, the simplest way to put something into the background is to start another Node process. It may seem like starting a process is heavyweight compared to a thread in other languages, but if you are doing something CPU intensive the cost of the exec() call is tiny compared to the rest of the work you are doing. It will be lost in the noise.

To be really useful, we don't want to just start a child process, but actually communicate with it to give it work. The childprocess module makes this very easy. childprocess.fork() takes the path to another script file and returns an event emitter. We can send messages to the child through this emitter and listen for responses. Here's a simple class I created called Workman to manage the process.

var Workman = {
    count: 4,
    chs:[],
    init: function(chpath, cb, count) {
        if(typeof count == 'number') this.count = count;
        console.log("using thread count", this.count);
        for(var i=0; i<this.count; i++) {
            this.chs[i] = child.fork(chpath);
            this.chs[i].on('message',cb);
        }
    },
    sendcount:0,
    sendWork: function(msg) {
        this.chs[this.sendcount%this.chs.length].send(msg);
        this.sendcount++;
    }
}

Workman creates count child processes, then saves them in the chs array. When you want to send some work to it, call the sendWork function. This will send the message to one of the children, round robin style.

Whenever a child sends an event back, the event will be handed to the callback passed to the workman.init() function.

Now that we can talk to the child processes it's time to do some drawing.

Parent Process

This is the code to actually talk to the screen. First the setup. pv is a new PixelView object. A PixelView is like an image view, but you can set pixel values directly instead of using a texture from disk. w and h are the width and height of the texture in the GPU.

var pv = new amino.PixelView().pw(500).w(500).ph(500).h(500);
root.add(pv);
stage.setRoot(root);

var w = pv.pw();
var h = pv.ph();

Now let's create a Workman to schedule the work. We will submit work for each row of the image. When work comes back from the child process the handleRow function will handle it.

var workman = Workman;
workman.init(__dirname+'/mandle_child.js',handleRow);
var scale = 0.01;
for(var y=0; y<h; y++) {
    var py = (y-h/2)*scale;
    var msg = {
        x0:(-w/2)*scale,
        x1:(+w/2)*scale,
        y:py,
        iw: w,
        iy:y,
        iter:100,
    };
    workman.sendWork(msg);
}

Notice that the work message must contain all of the information the child needs to do it's work: the start and end values in the x direction, the y value, the length of the row, the index of the row, and the number of iterations to do (more iterations makes the fractal more accurate but slower). This message is the only communication the child has from the outside world. Unlike with threads, child processes do not share memory with the parent.

Here is the handleRow function which receives the completed work (an array of iteration counts) and draws the row into the PixelView. After updating the pixels we have to call updateTexture to push the changes to the GPU and screen. lookupColor converts the iteration counts into a color using a look up table.

function handleRow(m) {
    var y = m.iy;
    for(var x=0; x<m.row.length; x++) {
         var c = lookupColor(m.row[x]);
         pv.setPixel(x,y,c[0],c[1],c[2],255);
    }
    pv.updateTexture();
}

var lut = [];
for(var i=0; i<10; i++) {
    var s = (255/10)*i;
    lut.push([0,s,s]);
}
function lookupColor(iter) {
    return lut[iter%lut.length];
}

Child Process

Now let's look at the child process. This is where the actual fractal calculations are done. It's your basic Mandelbrot. For each pixel in the row it calculates a complex number until the value exceeds 2 or it hits the maximum number of iterations. Then it stores the iteration count for that pixel in the row array.

function lerp(a,b,t) {
    return a + t*(b-a);
}
process.on('message', function(m) {
    var row = [];
    for(var i=0; i<m.iw; i++) {
        var x0 = lerp(m.x0, m.x1, i/m.iw);
        var y0 = m.y;
        var x = 0.0;
        var y = 0.0;
        var iteration = 0;
        var max_iteration = m.iter;
        while(x*x + y*y < 2*2 && iteration < max_iteration) {
            xtemp = x*x - y*y + x0;
            y = 2*x*y + y0;
            x = xtemp;
            iteration = iteration + 1;
        }
        row[i] = iteration;
    }
    process.send({row:row,iw:m.iw,iy:m.iy});
})

After every pixel in the row is complete it sends the row back to the parent. Notice that it also sends an iy value. Since the children could complete their work in any order (if one row happens to take longer than another), the iy value lets the parent know which row this result is for so that it will be drawn in the right place.

Also notice that all of the calculation happens in the message event handler. This will be called every time the parent process sends some work. The child process just waits for the next message. The beauty of this scheme is that Node handles any overflow or underflow of the work queue. If the parent sends a lot of work requests at once they will stay in the queue until the child takes them out. If there is no work then the child will automatically wait until there is. Easy-peasy.

Here's what it looks like running on my Mac. Yes, Amino runs on Mac as well as Linux. I mainly talk about the Raspberry Pi because that's Amino's sweet spot, but it will run on almost anything. I chose Mac for this demo simply because I've got 4 cores there and only 1 on my Raspberry Pi. It just looks cooler to have for bars spiking up. :)

This code is now in the aminogfx repository under demos/pixels/mandle.js.