Node.js TypeScript #13. Sending data between Worker Threads

JavaScript Node.js

This entry is part 14 of 15 in the Node.js TypeScript

The Worker Threads give us quite exciting features when compared to, for example, child processes. In this article, we create our instances of the MessageChannel to communicate with the Worker Threads. Aside from that, we also share the data between them: it’s something that is not possible with child processes.

Communicating using the MessageChannel

There are a few ways to communicate with Node.js Worker Threads. In the previous part of this series, aside from passing the data through  , we’ve used the   to send the data from the worker thread to the parent.

worker.ts

main.ts

In the example above we pass two different paths, both to   and  . This is connected to the fact that we can’t as of now create worker threads purely from TypeScript files. For more information check out the previous part of this series.

While this is a perfectly fine solution, there is more to it. We can create our ports if we find ourselves in a situation requiring a more complex solution.

Creating a new MessageChannel

A surprising fact is that the MessageChannel does not have any methods on its own. The only two properties it has are   and  , both instances of MessagePort.

Each one of the ports represents one end of the communication channel. Since the   is also an instance of the MessagePort, we can guess that our new ports also have the   function.

The MessageChannel is capable of working without involving any Worker Threads, as in the example above. To fully understand their purpose, let’s use them with a worker.

Sending MessagePort through a transferList

To set up such communication, we need to send one of the ports to the other thread, while listening to messages on the other end.

main.ts

worker.ts

Data.ts

 

Well, almost there! Running the above code results in an error:

MessagePort was found in message but not listed in transferList

Different ways of sending the data

The   that the error mentions is an additional argument of the   function. The above is connected to how the  function works. By default, it creates a clone of the data that we send, but we can change that behavior with the  . This is enforced if we use MessagePort. Let’s improve the example from above:

Sending MessagePort through transferList

main.ts

Now we explicitly state that we don’t want a clone of the  . Thanks to that, we pass an original object to the worker.

The MessageChannel might prove to be useful with more complex algorithms where we create multiple threads and would like to message the main thread from one of our nested Worker Threads.

Sending ArrayBuffers

In the previous examples, we send elementary data like a number. Nothing restraints us from sending more complex data structures.

It works without issues, but in the example above the   array is cloned. The more complex the data structure, the more computing power it takes to clone it.

We can prevent that from happening. To do that, we need to use the ArrayBuffer that differs a bit from the Buffer that we cover in the third part of this series. One of the easiest ways to create it is to use an Uint8Array that represents an array of 8-bit unsigned integers.

worker.ts

As you can see, the ArrayBuffer buffer that we need is in the   property. Since we operate on bytes here, we need to be aware of the restraints. Let’s look at how the number 256 is represented in binary:

2⁸  = 256

When we look at the example above we can figure out that the number 256 does not fit into eight bit.Uint8Array ignores excess data and due to that, 256 equals 0. You can use Uint16Array instead, but keep in mind that every element of that array uses more memory.

A crucial thing is that when we transfer the buffer and append it to the , it is no longer available to the sender.

TypeError: Cannot perform Construct on a neutered ArrayBuffer

The above is a mechanism implemented to guarantee no race conditions between 2 different threads trying to access the same chunk of memory. It is a fundamental issue with multithreading that is tackled in multiple ways in the multithreading world – for example with locks.

Sharing the data using SharedArrayBuffer

The above restraint, while preventing us from destroying our data, has a workaround. When using Worker Threads, we can designate a piece of memory accessible for more than one thread. To do that, we need to create Uint8Array using a special type of a buffer called SharedArrayBuffer.

This way the array is neither cloned, nor it is unavailable to the sender. We need to be especially careful though we stripped ourselves from the built-in mechanism that keeps the integrity of the data. You risk changing a value in one thread and expecting it to be something else in another thread.

Sharing arrays can come in handy, but in TypeScript, we use lots of objects. Currently, ECMA Script does not contain shared objects, but we can imitate similar behavior using DataView.

Summary

In this article, we dived deeper into Node.js Worker Threads, focusing on how can we communicate between Worker Threads and send data. We used MessageChannel to create a two-way communication channel between Node.js Worker Threads. Aside from that, we implemented different approaches to sending data between threads. It included cloning, sharing the data between multiple threads and restricting the access to just one thread through the  .

Series Navigation<< Node.js TypeScript #12. Introduction to Worker Threads with TypeScriptNode.js TypeScript #14. Measuring processes & worker threads with Performance Hooks >>
Subscribe
Notify of
guest
2 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Leo Plaw
3 years ago

Thank you for this. Most articles about worker threads are quite simple and do not deal with the broader scope of features. This has answered an important question for me, what exactly can be shared between the parent and worker threads.

But can you please explain under what circumstances that you would use the MessageChannel vs parentPort and workerPort?

Last edited 3 years ago by Leo Plaw
Leo Plaw
3 years ago

I’ve been doing a bit more research. It would seem that the MessageChannel is used to create a communication path between workers.

https://youtu.be/P1sWw1bLyVg?t=1428