MQTT in LabVIEW

I’ve been experimenting lately with MQTT in LabVIEW. MQTT is an open standard for message-based communications using a publisher/subscriber paradigm. It’s commonly used in the Internet of Things (IoT) world.

I’m working on updating an old LabVIEW application that makes heavy use of DataSocket, an NI-proprietary data communications mechanism. DataSocket has been deprecated, so my customer was interested in updating the communications scheme.

My friend Jörg Hampel of Hampel Software Engineering recommended I look into François Normandin’s LabVIEW Open Source MQTT client and broker. I started by watching François’ YouTube video, which gives a good introduction:

In parallel, I downloaded and installed the software with VIPM so I could follow along with the video. (Just search for MQTT in VIPM.)

François has created a powerful, complete MQTT solution, but makes it easy to get started by including templates that you can drag from the palette menus directly on your diagram. I actually tried several of them, so I could study their differences. For the most part, they are all very similar. For my simple application, I use a typical set of “start up” and “shut down” VIs which surround my communications.

(Click the image to enlarge.)

In between, I can call VIs to publish data and/or wait for incoming data from my subscriptions with an event structure.

The application I am replacing tends to be very chatty. It consists of three components (an HMI and two headless DAQ systems) which share state information, setpoints, measurement results, watchdog counters, and error information through DataSocket. Each has VIs that constantly (about once every quarter second) read and write the values of about 40 scalar global variables. It’s not elegant, but it works.

My first attempt at replacing this with MQTT was a straight replacement of DataSocket Write with MQTT Publish, and DataSocket Read with events in the event structure. I put the Publish calls in the Timeout case with a 250ms timeout. While this initially worked in my testing with two MQTT clients, I soon realized that I’d made a silly mistake by doing it this way.

Once the three components were up and running, there was enough chatty communication coming from the HMI that the headless DAQ systems never timed out, so they never published any data. To solve this, I split up my loops–one for publishing and one for subscribing, and everything started working again.

As an aside, I also decided to make the application less chatty–something I could have done with DataSocket, too. Instead of constantly sending everything, I now only communicate setpoints when they are set, and only send results when they are generated. I have taken or plan to take a similar approach with the other data; there’s no reason for it to be chatty about anything but watchdog counters.

I should also mention that I tried three different brokers. (The broker is at the center of the communications, and it’s the piece that makes sure that data gets from a publisher to all of the subscribers.)

François supplies his own native LabVIEW broker which works very well. I didn’t find much in the way of debugging tools to know what the broker was doing (“what values does it see?”, “who is connected?”), so I started looking for other options.

Jörg suggested I look at ActiveMQ from Apache.org, which supports the MQTT protocol (among others). The nice thing about it is that it has a console web page you can use to see what the broker is doing. Unfortunately, I routinely managed to get it into a state where my MQTT client code wouldn’t stay connected for more than a second or two at a time. I was frustrated and almost abandoned MQTT at this point, but I decided to look for a different broker.

The definitive MQTT broker is called Mosquitto, from the Eclipse Foundation. I installed it on a Linux VM, and things immediately worked so much better. I’m not sure what the incompatibility was with ActiveMQ, but I’m glad I didn’t abandon this MQTT project. Mosquitto doesn’t have an interactive web page for diagnostics, but you can put it into verbose mode to see what the broker is doing. This was sufficient for me–especially as I worked to make the application less chatty and as I tested killing and restarting clients.

Once I had things working with Mosquitto, I also went back and tried the native LabVIEW MQTT broker, and confirmed that it still worked. I’ve also done a little bit of memory monitoring, to try to ensure that the applications don’t have memory leaks using these libraries.

In the end, the conversion from DataSocket to MQTT was several times more complicated than I thought it would be. I had a lot of early success, but as they say, the devil is in the details. I was learning a new protocol, a new API and library, using brokers I had no prior experience with. Combine this with an application that isn’t very disciplined with its communication, and it’s not a surprise that it took a few extra days to resolve. I will also add that the time I spent coping with an unreliable broker encouraged me to be rigorous in handling connection errors and quickly reconnecting.

A big thanks to Jörg Hampel and François Normandin for their encouragement and wisdom, and especially to François for his huge contribution of creating and maintaining the MQTT LabVIEW open source client and broker projects.