Ceci est une ancienne révision du document !
Hi everyone. It’s been quite a while since I’ve been able to do much in the way of writing, but things are slowly getting better, so I’ve decided to start again with a discussion of MQTT. MQTT is a lightweight, easy to use, network protocol to send data from one computer or application to another. It works in a client/server model, using a broker, publishers and subscribers. So what does that mean? One of the best examples I found that helps to break it all down, was to compare it to a post office (https://www.baldengineer.com/mqtt-introduction.html).
The post office (broker or server) exists to route letters and packages (messages) from people and companies (publishers) to other people and companies (subscribers). If you want to send a message to someone, you create the message and send it to someone else through the post office. The post office receives your message and routes it to the recipient. However, the recipient needs to be known to the post office for them to be able to deliver it to them. In the above example, Mike wants to send a message to Sally. Mike (the publisher) sends the message to the post office and they know Sally's address, so they send it to her (subscriber). Sally can also send messages (not only a subscriber, but a publisher as well), through the post office, to John (another subscriber).
In the real world, we have an application, acting as a publisher, that sends messages, based on a topic, to the broker. The broker then sends the message along to any other program, on any other machine or the same machine, that has subscribed to that message topic. I have been asked, what happens if one of the clients that are subscribed to a topic goes away for some reason. There are three Quality of Service (QoS) levels that can be set for any message being sent to the broker. When a client normally connects to the broker, it is in a non-persistent manner and the QoS level is set to 0, so any messages that arrive when a subscribing client is not connected are lost. If it is important that the subscribers get all messages, the publishing client must set up a persistent session and send the messages with a Quality of Service level higher than 0. We’ll deal with QoS and persistent sessions next month when we create our publishing client.
Let's assume that we have a Raspberry Pi 3 that sits in our garage that reads a DHT11 or DHT22 Humidity and Temperature sensor. Let's further assume that we have another Raspberry Pi 3 that we want to use to monitor that information in our bedroom. We can set up a MQTT broker on our bedroom RPi to receive the messages by using Mosquitto available in the software manager. The RPi in the garage can use a modified version of the software that we used way back in Full Circle Magazine #109 to monitor the sensor and publish the messages to the Mosquitto broker in the bedroom. On the RPi in the bedroom, we will write a simple Python program to subscribe to those messages. But first, let’s look deeply into MQTT under Python. For testing purposes, we’ll run the sensor, publishing program, broker and subscriber program all on the same Raspberry Pi.
The first thing we need to do is install Mosquitto and its two clients. From the Main Menu, go to Preferences | Add / Remove Software. In the search box, type ‘mosquitto’. You are looking for two packages. The first is the broker. It shows up as ‘MQTT version 3.1 / 3.1.1 compatible message broker mosquitto-1.4.15-0mosquitto1’. Select it, then, a few lines down from that, you want to find ‘Mosquitto command line MQTT clients mosquitto-clients-1.4.15-0mosquitto1’. Select this as well, then click the ‘Apply’ button. Once these are installed, you can dismiss the Software manager. Alternately, you can use apt-get to install it. In a terminal window, type sudo apt-get install mosquitto mosquitto-clients and they will be installed for you.
Once you have these programs installed, reboot the RPi and mosquitto will automatically be started. You won’t see anything, but it is running. Now we’ll make sure things are working correctly. Open two terminal windows. Put one on the left side of the screen and the other on the right side. In the left window type mosquitto_sub -h localhost -t test When you press <enter>, nothing appears to happen, but you’ve created your first MQTT subscription. Let’s press pause for a second and see what we did. The first part of the line we just typed was (of course) the command to run ‘mosquitto_sub’, the command line subscription client for MQTT. The ‘-h localhost’ tells the client that the broker, or host, is running on our local host. The ‘-t test’ tells the client that it should subscribe to the topic named ‘test’. More on that in a minute. The reason that nothing happened when you pressed <enter> is that it’s waiting for a message to be published on the ‘test’ topic. So, let’s move over to the right terminal window and type
mosquitto_pub -h localhost -t test -m 'Hi There' Now look at the left terminal. You should see ‘Hi There’ echoed in the terminal. The ‘mosquitto_pub’ is the command-line publishing client. It runs one command at a time, so, everytime you want to publish a message, you have to enter the entire command into the terminal. Just like the command-line for the subscription client, the ‘-h localhost -t test’ portion of the command says to attach to the broker and subscribe to the ‘test’ topic. The final part of the command is the message itself, which is, of course, the ‘-m “Hi There”’ portion. While this seems very simple on the face of it, there is a lot going on here. The mosquitto broker (server) is running on our RPi and sits there by default listening on TCP/IP port 1883. You can change the port number to most any available TCP/IP port, however port 1883 is registered with IANA specifically for MQTT. TCP/IP port 8883 is also registered, for using MQTT over SSL. If you don’t want to run your own broker, there are many public brokers you can use, some of them are free. You can find a list of some of these and their conditions for use at https://github.com/mqtt/mqtt.github.io/wiki/public_brokers.
When a program wants to publish to a broker, it first needs to connect, then sends the topic and the message. It doesn’t have to be concerned who’s there, if anyone. A subscribing program also doesn’t have to care about much, other than subscribing to the proper topic that is being published on the broker. The key here is that if you accidentally subscribe to the wrong topic, nothing will ever come in. When we set up our two terminal window test above, we started the subscriber client before the first message ever came in. The broker didn’t know (or care) what the topic was that we were subscribing to. It assumed we knew what we were doing. In the same way, when we published our ‘Hi There’ message on the ‘test’ topic, the broker simply took it and sent it out to whatever clients out there that happened to be subscribed to the topic, if any.
The topic itself can be as simple as ‘test’ or extremely verbose. Some basic guidelines for topics are: • Case sensitive • Must be a UTF-8 string • Must consist of at least one character to be valid. • Wildcards are allowed when subscribing to a topic hierarchy, but when publishing to a topic, any message can be published to only a single topic (no wildcards are allowed when publishing). • There are no topics created on a broker by default other than $SYS system topics. • Topics are created by a subscribing or publishing client. They are NOT permanent. • Topics should not start with ‘$’, since those messages are for broker system messages. • Topics should be specific in nature, not general.
Other than that, there are very little constraints on what a topic is or how it is constructed. Typically, if a client needs to publish more than a single type of message, the topics will be level based with each level separated by a forward slash (/). For example, in the program we will be writing to monitor and publish the DHT sensor values, we should use two different topics. One for the humidity value and one for the temperature value. We could combine the values into a string and only use one topic, but in this case, we will publish them separately. Making a very gross assumption, we will provide for a complete home solution, with multiple sensors of various types in multiple rooms of the house. We could then start our topic(s) with: housesensors/
This describes the topic to be one that will have sensor data specific to the house itself. This could include bedrooms, living areas and garden areas, but not something like a parking lot. The next level could possibly be floor based, as in first floor, basement and so on. housesensors/firstfloor/ Next could be the room or area. housesensors/firstfloor/bedroom/ Finally, we could set the topic to the specific type of value being published. housesensors/firstfloor/bedroom/humidity housesensors/firstfloor/bedroom/temperature
Notice that none of the topic strings have spaces in them. While spaces are not specifically prohibited, UTF-8 allows for many kinds of white space and could confuse things. You could also use mixed cases like ‘HouseSensors/FirstFloor/Bedroom/Humidity’, but since all topic strings are case sensitive, if you mistype a topic when publishing or subscribing, it could cause a good amount of time to debug things. I feel it makes it easier to go with all lower case. For our Python programs to communicate MQTT, we will use the paho-mqtt library. You can install it using pip. For Python 2.x: pip install –user paho-mqtt Or for Python 3.x: pip3 install –user paho-mqtt Full documentation on the phao library can be found at https://www.eclipse.org/paho/clients/python/ .
As a sample program for you to test with, I’ll give you a short program that you can run and use the mosquitto_pub to send messages to it based on sample code from the eclipse site. I’ve made a couple of changes to make it work properly here. Of course, we need to import the library to get started. import paho.mqtt.client as mqtt Next, we’ll create a callback function (shown top right) that will run whenever the client connects to the broker. Next, we will make a callback function that runs whenever a message is received from the broker. It will print in the terminal the topic followed by the message. def on_message(client, userdata, msg): print(msg.topic+“ ”+str(msg.payload)) Now we instantiate the client and bind our callback functions to it.
client = mqtt.Client() client.on_connect = on_connect client.on_message = on_message Next, we connect to the broker, located at localhost. The 60 at the end of the call specifies a ‘keepalive’ time of 60 seconds, where the client pings the broker every 60 seconds. client.connect(“localhost”, 1883, 60) Finally we tell the client to loop_forever. Use the <ctrl>C keyboard interrupt to exit the sample program. # Blocking call that processes network traffic, dispatches callbacks and # handles reconnecting.
# Other loop*() functions are available that give a threaded interface and a # manual interface. client.loop_forever() Now, save the program as ‘client1.py’ and run it. Go back to the terminal running mosquitto_pub and republish the previous message to the ‘test’ topic. You should see it appear in the terminal window of our running Python program. I’ve put the code up on Pastebin at https://pastebin.com/zL5ed9g9 Next month, we’ll update our original Python program from Full Circle Magazine #109 to support MQTT. Until then, have fun.