Message Queuing Telemetry Transport (MQTT) Protocol

Introduction to MQTT Protocol

MQTT Protocol is a lightweight, publish-subscribe, machine to machine network protocol for message queue / message queuing service. It is designed for connections with remote locations that have devices with resource constraints or limited network bandwidth, such as in the Internet of Things (IoT). It must run over a transport protocol that provides ordered, lossless, bi-directional connections—typically, TCP/IP. It is an open OASIS standard and an ISO recommendation (ISO/IEC 20922).

MQTT Protocol defines two types of network entities: a message broker and a number of clients. An MQTT broker is a server that receives all messages from the clients and then routes the messages to the appropriate destination clients. Clients can publish messages to topics or subscribe to topics to receive messages from other clients. MQTT Protocol supports three quality of service levels for message delivery: 0 – at most once, 1- at least once, 2 – exactly once. MQTT Protocol also supports persistent sessions, security features and various extensions for different use cases.

History of MQTT Protocol

The MQTT protocol was invented in 1999 by Andy Stanford-Clark of IBM and Arlen Nipper of Cirrus Link.

MQTT is a lightweight, publish-subscribe network protocol that was first developed in the late 1990s by Dr. Andy Stanford-Clark of IBM and Arlen Nipper of Cirrus Link Solutions (Previously in Arcom). The MQTT protocol was designed to be used for monitoring and control of M2M (machine-to-machine) and IoT (Internet of Things) devices over low-bandwidth, high-latency networks.

MQTT was originally used in the oil and gas industry to monitor remote drilling rigs and other industrial processes, but it has since been adopted for a wide range of other applications, including home automation, healthcare, and smart cities.

In 2013, the MQTT specification was submitted to the OASIS standards organization and became an official standard in 2014. Today, MQTT is widely used in a variety of industries and is supported by a large number of IoT devices and platforms.

Features of MQTT Protocol

• Lightweight and efficient to minimize resources required for the client and network bandwidth.
• Enables bidirectional communication between devices and servers. Also, enabling broadcasting messages to groups of things.
• Scales to millions of things.
• Specifies Quality of Service (QoS) levels to support message reliability.
• Supports persistent sessions between device and server that reduces reconnection time required over unreliable networks.
• Messages can be encrypted with TLS and support client authentication protocols.

Structure of MQTT Protocol

MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe network protocol for message queue service. It is designed for connections with remote locations that have devices with resource constraints or limited network bandwidth, such as in the Internet of Things (IoT). It must run over a transport protocol that provides ordered, lossless, bi-directional connections—typically, TCP/IP.

The MQTT protocol defines two types of network entities: a message broker and a number of clients. An MQTT broker is a server that receives all messages from the clients and then routes the messages to the appropriate destination clients. The clients communicate with the broker using a command and command acknowledgement format. That means each command has an associated acknowledgement.

The MQTT packet or message format consists of a 2 byte fixed header (always present) + Variable-header (not always present)+ payload (not always present). The fixed header field consists of the control field and the variable length packet length field. The control field is an 8 bit field that contains all of the protocol commands and responses. It is divided into two 4 bit fields ,and contains all of the protocol commands and responses. The first 4 Most significant bits are the command or message type field and the other 4 bits are used as control flags. The packet length field is of variable length between 1 and 4 bytes. Each byte uses 7 bits for the length with the MSB used as a continuation flag. The remaining length is the number of bytes following the length field, includes variable length header and payload.

The variable header contains additional information specific to certain message types, such as topic names, client IDs, user names, passwords etc. These are encoded as UTF-8 strings. The payload contains binary data that is application specific. The content and format of this data depends on what type of message it belongs to.

Different Technical Terminologies used in MQTT

1. Clients: Devices or applications that publish or subscribe to messages. Clients can be a sensor, actuator, mobile device, or any other device that can transmit or receive data over a network.
2. Topics: A topic is a string that represents a specific subject or channel of communication. Clients publish messages to a specific topic and subscribe to receive messages on specific topics.
3. Broker: The broker is the central component of the MQTT architecture. It acts as a message router, receiving messages from clients and forwarding them to the appropriate subscribers.
4. Publishing: When a client wants to send a message, it publishes the message to a specific topic on the broker. The broker then distributes the message to all subscribers that have subscribed to that topic.
5. Subscribing: When a client wants to receive messages on a specific topic, it subscribes to that topic on the broker. The broker then forwards any messages published to that topic to the subscriber.
6. Quality of Service (QoS): MQTT supports three levels of Quality of Service (QoS) to ensure the reliability of message delivery. The three levels are:

• QoS 0: At most once delivery. The message is delivered at most once, with no guarantee of delivery.
• QoS 1: At least once delivery. The message is delivered at least once, with a guarantee of delivery.
• QoS 2: Exactly once delivery. The message is delivered exactly once, with a guarantee of delivery.

How Does MQTT Works?

MQTT operates on a publish-subscribe model, where devices can send and receive messages without the need for direct communication. Devices are organized into topics, and messages are sent to specific topics. Devices that subscribe to a particular topic receive messages sent to that topic. This allows devices to send and receive messages in a scalable and flexible manner, making it well-suited for use in large, complex IoT networks.

MQTT also includes a number of important features that make it well-suited for use in IoT networks. For example, it includes an “at most once” delivery mechanism that ensures that messages are delivered at least once, and the protocol includes a number of quality of service (QoS) levels that allow for different levels of message reliability.

• A client connects to a broker using TCP/IP and optionally provides authentication credentials (such as username and password).
• The client sends a CONNECT packet to the broker with its client ID (a unique identifier) and other connection parameters (such as keep-alive interval, clean session flag, etc.).
• The broker responds with a CONNACK packet that indicates whether the connection was accepted or rejected.
• The client can then publish messages to topics using PUBLISH packets with various options (such as QoS level, retain flag, etc.). The broker acknowledges the receipt of each message depending on the QoS level requested by the publisher.
• The client can also subscribe to topics using SUBSCRIBE packets with QoS levels indicating how reliably it wants to receive messages from those topics. The broker responds with SUBACK packets that confirm the subscriptions and QoS levels granted by the broker.
• The broker delivers messages from publishers to subscribers based on their topic subscriptions and QoS levels using PUBLISH packets with various options (such as DUP flag, packet ID, etc.). The subscriber acknowledges the receipt of each message depending on the QoS level granted by the broker.
• The client can unsubscribe from topics using UNSUBSCRIBE packets with topic names. The broker responds with UNSUBACK packets that confirm the unsubscriptions.
• The client can disconnect from the broker using DISCONNECT packet when it no longer needs communication.

How to use MQTT?

To use MQTT in your own projects, you will need three components:

• An MQTT client library that implements the protocol on your device or platform.
• An MQTT broker that acts as an intermediary between clients.
• An MQTT application that defines your business logic.

There are many open source or commercial options available for each component depending on your needs and preferences.

• MQTT client library: Paho , Eclipse Mosquitto , HiveMQ , or AWS IoT Core etc…
• MQTT broker: You can use Eclipse Mosquitto , HiveMQ , VerneMQ , EMQ X , or AWS IoT Core.
• MQTT Server: You can use Node-RED , Home Assistant.

Applications of MQTT Protocol

1. IoT Sensor Networks: MQTT is commonly used to connect IoT devices, such as sensors and actuators, to the cloud. It enables the devices to transmit data to a central server or database, which can then be used for monitoring, analytics, and decision-making.
2. Industrial Automation: MQTT is also used in industrial automation systems, where it is used to collect and transmit data from sensors and control devices. This data can be used for monitoring and controlling industrial processes, such as manufacturing and supply chain management.
3. Home Automation: MQTT can be used in home automation systems to enable communication between smart devices, such as thermostats, security systems, and lighting. This allows users to control their home devices remotely and automate certain tasks.
4. Mobile Applications: MQTT can be used in mobile applications to enable real-time communication between mobile devices and servers. This is useful for applications that require real-time updates, such as chat apps, gaming apps, and location-based services.
5. Healthcare: MQTT is also used in healthcare applications to monitor patient health and collect data from medical devices. This data can be transmitted to healthcare providers in real-time, allowing for more efficient and accurate healthcare delivery.

Advantages of MQTT Protocol

MQTT protocol has several advantages that make it a popular choice for IoT applications:

1. Lightweight: MQTT was designed to be a lightweight protocol, making it well-suited for resource-constrained devices, such as sensors and actuators, that are commonly used in IoT applications.
2. Efficient: MQTT uses a publish-subscribe model, which enables efficient communication between clients. The broker acts as a message router, distributing messages to the appropriate subscribers, which reduces the amount of data that needs to be transmitted over the network.
3. Reliable: MQTT supports three levels of Quality of Service (QoS), allowing clients to choose the level of reliability that they need for their specific use case.
4. Secure: MQTT supports secure communication using Transport Layer Security (TLS) encryption, which helps to protect against eavesdropping and tampering of data in transit.
5. Scalable: MQTT can handle large numbers of clients, making it suitable for large-scale IoT deployments.
6. Interoperable: MQTT is an open standard, which means that devices from different manufacturers can communicate with each other using MQTT, providing greater interoperability and flexibility for IoT solutions.
7. Simple: MQTT has a simple and well-defined protocol, which makes it easy for developers to implement and integrate into their applications.

Disadvantages of MQTT Protocol

Like any technology, MQTT protocol has some disadvantages, though it has lot of advantages:

1. Limited data types: MQTT only supports simple data types, such as strings and integers, which may not be sufficient for some applications that require more complex data structures.
2. Lack of Quality of Service (QoS) guarantees: Although MQTT supports three levels of QoS, the guarantees provided by each level may not be sufficient for some applications.
3. Single broker architecture: MQTT uses a single broker architecture, which can create a bottleneck in terms of performance and scalability if the broker becomes overwhelmed with traffic.
4. Security vulnerabilities: While MQTT supports encryption, it does not provide end-to-end security, which means that messages can still be intercepted or altered between the client and the broker.
5. Lack of built-in message persistence: MQTT does not provide built-in message persistence, meaning that messages are not stored if they are not delivered. This can be problematic for applications that require reliable message delivery.

Future Development and Enhancement of MQTT Protocol

MQTT (Message Queuing Telemetry Transport) is a widely used protocol for communication between Internet of Things (IoT) devices and servers. MQTT is known for its lightweight nature and low bandwidth requirements, making it ideal for use in resource-constrained environments. As technology continues to evolve, there are several potential developments and enhancements that could be made to the MQTT protocol.

1. Support for More QoS Levels: Currently, MQTT supports three QoS (Quality of Service) levels, namely QoS 0, QoS 1, and QoS 2. However, there is a need for more granular control over message delivery, especially in high-stakes environments. One potential enhancement could be the addition of more QoS levels that provide more robust message delivery guarantees.
2. Support for Additional Data Types: Currently, MQTT only supports a limited set of data types, such as integers, floats, and strings. As IoT use cases continue to expand, there is a need for additional data types to support more complex data structures. One potential enhancement could be the addition of support for binary data and custom data types.
3. Better Security Features: While MQTT already provides basic security features, such as TLS encryption, there is always a need for more robust security measures, especially in environments where data privacy is critical. One potential enhancement could be the addition of support for advanced encryption algorithms and multi-factor authentication.
4. More Efficient use of bandwidth: MQTT is known for its lightweight nature, but there is still room for improvement in terms of bandwidth efficiency. One potential enhancement could be the addition of support for delta compression, which only sends the changes in data values rather than the entire payload.
5. Support for Offline Communication: Currently, MQTT requires devices to be connected to the network to receive messages. One potential enhancement could be the addition of support for offline communication, which would allow devices to receive messages even when they are disconnected from the network.
6. Integration with Other Protocols: MQTT is already widely used, but there is always a need for better integration with other protocols. One potential enhancement could be the addition of support for integration with other IoT protocols, such as CoAP and HTTP.

MQTT Related Topics

• MQTT Protocol in IoT: The MQTT protocol in IoT plays a crucial role in enabling efficient and reliable communication between connected devices, facilitating seamless data exchange in smart homes, industrial automation, and other applications.
• MQTT Protocol Port: The MQTT protocol port, typically set to 1883 for unsecured connections and 8883 for secured connections, ensures efficient and secure data transmission between IoT devices and brokers in a wide range of applications.