Mosquitto, the MQTT broker – Under the Hood Part2

This is the continuation of the Mosquitto MQTT part 1 blog.

To recap, we have set up the MQTT broker (Mosquitto) and verified the messages transferred between the broker and the client based on a topic listened by the broker.

Now lets go deeper to understand more about the protocol and packet transfers involved by capturing the network traffic and analysing it. I used wireshark (my goto tool for whatever may be the network problem) on the ubuntu mate 16.04 machine to capture the network traffic.

To reduce the number of packets which we need to analyse let us add wireshark filter to check only packets that is using MQTT designated port i.e port 1883 (pls refer to part 1 for more details). Since this is a TCP connection the exact filter which i used is “tcp.port == 1883“.

As we can see the total traffic on port 1883 is just 12 packets. The first three packets are for establishing the TCP connection and last two are used to tear down the connection.

MQTT Header — Few words

MQTT fixed header is 2 byte (16 bits) and it is split up into four fields:

4 bits for MQTT standard message types (16 possible )
1 bit for duplicate flag.
2 bits for QoS.
1 bit for retain information.
8 bits for remaining length mentioned in bytes.

For more info, go through the below link which is very simple and very elaborate:

http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#msg-format

Connect Packet — The beginning

Now let us directly jump into packet no 81, wireshark shows the protocol type of this packet as mqtt and we are going to discuss packets of this type only. Let us open up the packet to see more details.This is connect request which is sent from the client to the machine where MQTT broker is running.

In packet number 81, mqtt message type field has the value set to 1 (binary) which corresponds to connect command.Rest of the fields are not set. The payload has details about the connect flags. This is variable header and in this field only clean session flag is set which means broker must discard any previously maintained information about the client and treat the connection as clean. The broker must also discard any state when the client disconnects. In other words the connection should be treated as a new connection. And also the keep alive timer is set to 60 seconds followed by the client id.

Connect Packet — Acknowledgement

Next let us check packet number 83, which is connect ack packet from the broker/server. This packet also starts with fixed 2 byte header containing the details like message/command type, duplicate, QoS and message len fields. Fixed header’s message type field set to 10 (binary) which refers to connection acknowledgement. This has the handshake information about the connection.There are 6 possible return code types at the time of this writing. Pls note that there is no payload for this packet as it is just an acknowledgement packet.

Publish — Message transfer

Next is packet number 85, which is publish message containing the actual message which we are sending from the client machine. The packet starts with fixed header’s message type field set to 11 (binary) which corresponds to publish message. The topic field points to the actual topic to which the we are publishing the message. In our case the topic is mqtt and the message is “Hello from 192.168.1.121” (Pls refer to Part 1).
Obviously this will vary depending upon the topic to which you are sending and the actual data you are sending.

Disconnect — Good day

Next is connection tear down request from the client, packet number 86. The packet starts with fixed header’s message type field set to 111 (binary) and rest of the fields are not set. For disconnect command there is no variable header and payload. After this packet the connection is closed. If any new messages needs to be published then the whole process is carried out again.

Our message was passed to the broker through the network by MQTT protocol by using just four packets (with the connection disconnect). If the fixed protocol header overhead is taken into account it comes to just 8 bytes which is pretty much acceptable.

The main advantage of this protocol is :

1. The fixed header is only 2 bytes.
2. Has QoS support at least at the basic level.
3. Timer to maintain the session.
4. Acknowledgements.
5. Built in intelligence to take decision about holding the client’s information.
I have tested this on multiple ARM platforms also and it worked as mentioned above.