Home > Blog > Why so salty? Local privilege escalation on SaltStack minions
Why so salty? Local privilege escalation on SaltStack minions
Mat Rollings, Vulnerable App Developer at Immersive Labs, has uncovered a command injection vulnerability in SaltStack's Salt programme.
It’s not every day you find malicious process names exploiting your code. On 5th November 2020, that’s exactly what our own Mat Rollings discovered – in the form of a command injection vulnerability in SaltStack’s Salt programme, a popular tool used to automate and secure infrastructure.
Salt’s usage is split into two roles: a master system responsible for controlling the systems connected to it, and the minions, which connect to the master system and respond to issued commands. Both roles are typically run as root.
The vulnerability allowed privilege escalation via specially crafted process names on a minion when the master called restartcheck. The vuln affects all versions of Salt between 2016.3.0rc2 and 3002.2. Here’s what Mat found.
While browsing the source code for a previously disclosed SaltStack vulnerability, I decided to run the code through Bandit, a security scanner for Python applications that catches issues. I expected a handful of results, as its codebase is relatively large and has existed for several years. What I didn’t expect was 117 high severity issues.
Now, it can take a significant amount of time to parse through all this data, with many of these issues returning as false positives or holding little importance. After noticing several instances of subprocess.Popen used in conjunction with shell=True in the codebase, it was time for me to focus on researching potential command injections.
A few turned out to be in no way controllable, yet I discovered one could be controlled via some clever trickery involving process names.
The minion’s restartcheck was vulnerable to command injection via a crafted process name, when the process has open file descriptors associated with (deleted) at the end of a filename. (Note, the leading space is required for the injection to function.) This allowed any local user to escalate their privileges to root, provided they were able to create files on the minion in a directory that was not explicitly forbidden.
The vulnerable code
The vulnerable code is at line 615 in restartcheck.py where subprocess.Popen is called with shell=True and a command that can be manipulated by an attacker:
Here, package is formed from the process name and cmd_pkg_query is one of the following, depending on the OS:
Debian: dpkg-query –listfiles
RedHat: repoquery -l
NILinuxRT: opkg files
If a Bash control character (such as ;, |, &&, etc) could be inserted into the process name, the injection could be triggered when this code is reached.
So how do you reach the code? A process needed to first have a filehandler open to a file with a name that ended in (deleted) and that resided in a directory that wasn’t explicitly forbidden.
The list of forbidden directories immediately ruled out some of the more obvious places, such as /tmp or /dev/shm. However, a low privileged user may have access to a few common ones, such as /var/crash, /var/spool/samba, or /var/www.
During my research, I found that process names are tricky to modify reliably because the name listed by ps may not be the same returned by the Python psutil library. In Linux, process names can contain any character except null. Any user can start a process on a system, and the process itself will set its own name. Process names are a good target for command injection vulnerabilities, as developers don’t expect them to include special characters.
Process names can be directly set using exec -a, yet this won’t work in busybox or sh shells and didn’t appear to show the same name when using psutil. Process names can also be modified by directly manipulating procfs, but this also led to inconsistent results.
The simplest and most consistent way to set a process name is to rename the binary or script being run. As filenames in Linux cannot contain forward slashes (/), the injectable commands were restricted. However, it was trivial to bypass this using base64 encoding.
For the exploit to work, I required a writable directory not explicitly forbidden by SaltStack. To find the directories that matched this restriction, I ran the PoC script with no arguments.
With the writable directories now at my disposal, I then passed one with the -w flag and a command with the -c flag to create a process that contained the command injection in the process name and an open file handler that would cause the exploit to be triggered when the master called restartcheck.
In order to demonstrate the vulnerability, I’ll go through how to perform the exploit, step by step, with the aim of creating a simple file as a root user. First, I ran the script providing the proper flags.
Now that I’d confirmed the malicious process was running with a command injection in the name and a filehandler open, I could issue the restartcheck.restartcheck command on the SaltStack master. Once complete, I could check for the existence of the hacked file in the root directory.
This is all well and good, but I was certain I could do something a bit more exciting, such as getting a shell as root. The video below shows one way of doing this, by copying the find binary and making it suid:
Here’s how the exploit looks from the master’s point of view, with the injection shown in the highlighted line.
Additional security considerations
As always, there were further security issues that needed to be addressed.
Containerized processes were listed on the host machine, which meant the exploit could be performed from within a container to gain command execution as root on the host machine.
Although unlikely, this injection may be performed by an attacker without local shell access. Under certain circumstances, remote users can influence process names.
The security fix for this issue was released on 4th February 2021. You can find out more about CVE-2020-28243 over on the SaltStack page here.
No one expects process names to be malicious. Developing and maintaining secure code is a difficult and arduous task, so you’ve got to be on the ball when developing. Are you confident your web applications are secure? Immersive Labs’ series on Python Secure Coding is the best place to practice and level up your skills in identifying, exploiting, securing and validating common vulnerabilities in web applications. Log in to your Immersive Labs account or book a demo for a tour of our human cyber readiness platform today.
Mat Rollings Vulnerable Application Developer, Immersive Labs @stealthcopter