Demystified: The DoD’s Automated Vulnerability Mining Tool

In late 2019, David Haynes, a security engineer at Internet infrastructure company Cloudflare, stared at a bizarre image and couldn’t calm down. “It was just messing around,” he recalls. “The machine generated a whole bunch of gray and black pixels.” He declined to release the images, citing the risk of security risks.

Haynes’ caution is certainly justified. The image is from a tool called Mayhem, whose job is to detect software and find potential security holes in it. This tool comes from Carnegie Mellon University, and the current project belongs to the start-up ForAllSecure. Haynes has been testing the tool on Cloudware software to continuously resize images to speed up the site and provide more sample photos for training. Maythem mutated these sample images by triggering a potential bug that caused the image processing software to crash, resulting in anomalous images. The potential bug has irritated users who bought Cloudflare products to keep their websites running smoothly.

Fields of application for Mayhem

Since then, Mayhem has become part of Cloudflare’s standardized security detection tool, which is also used by the U.S. Air Force, Navy and Army. Last month, the Pentagon awarded ForAllSecure a $45 million contract to expand the use of the tool across U.S. military agencies. It was clear that officials needed a tool that could find a large number of potential vulnerabilities. A government report released in 2018 found that nearly every weapon system the U.S. Department of Defense examined from 2012-2017 had critical software vulnerabilities.

Mayhem is not yet mature enough to fully replace the role of human debuggers — professionals who combine software design knowledge, code reading ability, creativity, and intuition to efficiently find development bugs. But ForAllSeucre co-founder and CEO David Brumley says the tool is positioned to help human experts become even more efficient. After all, human time and energy are limited, and the software security vulnerabilities in the world are constantly increasing, and more vulnerabilities are exposed to the world every minute.

“Security isn’t about being ‘safe,’ it’s about being unsafe,” Brumely said.

Mayhem grew out of an unusual hacking contest held in a Las Vegas casino ballroom in 2016. The stage was empty, save for seven computer servers in action. Each contestant hosts a software robot that attempts to find and exploit vulnerabilities in the opponent’s server, while working to fix software flaws in its own server. After eight hours of competition, Brumley’s team from Carnegie Mellon University’s Security Lab won the top prize of $2 million with Mayhem. The server they were using at the time was also moved into the Smithsonian.

Brumley, who is still a professor at Carnegie Mellon University, said the experience reinforced his confidence that what he learned in the lab could be of great use in the real world. He temporarily put aside digging into the robot’s attack capabilities, and instead focused on improving its security defense mechanism and promoting the commercial transformation of the results. He explained, “This cybersecurity challenge proves that fully autonomous security systems are not impossible. Computers can lead to good defensive performance.”

The Chinese and Israeli governments have also invested a lot of energy in this area. Both sides extended an olive branch to ForAllSecure, but the company ultimately decided to work with the U.S. government. The new technology represented by Mayhem will be quickly introduced to the U.S. military through a contract with the Pentagon’s Department of Defense Innovation Unit.

ForAllSecure was challenged to prove Mayhem’s capabilities, and Mayhem lived up to its expectations by automatically finding a security flaw in the control software of the U.S. military’s military-to-civilian airliner within minutes. The aircraft manufacturer then verified and fixed the problem.

Of course, Mayhem’s exploits don’t stop there: Earlier this year, it found a vulnerability in the OpenWRT software commonly used by thousands of network devices. Last fall, two of the company’s interns used Mayhem to find a software bug that allowed users to cast video from their phones to TVs, and won a bug bounty from Netflix.

Brumley noted that automotive and aerospace companies have shown strong interest in the tool. Today’s cars and airplanes are increasingly in need of software that needs to run in a secure, reliable, stable manner for years, preferably with minimal updates.

Mayhem’s Vulnerability Identification Pattern

Mayhem is currently only available for programs based on Linux operating systems, and provides two modes of vulnerability identification: broad scanning and targeted scanning.

The so-called extensive scanning uses a technique called “fuzzing”. It will randomly generate instructions or images, deliver them to the target software in batches, and identify whether there is a trigger condition that can cause a crash. The second method, also known as symbolic execution testing, is equivalent to creating a simplified mathematical representation of the target software. Through analysis of this doubly simplified model, potential weaknesses in the actual target can be identified. Brumley, co-founder and CEO of ForAllSecure, said that when it comes to identifying software vulnerabilities, “computers can do a great job.”

In recent years, fuzzing has been widely used in the field of computer security. Last year, Google released a fuzzing tool that it said had found more than 16,000 vulnerabilities in its Chrome browser. But Cloudflare’s Haynes said the technology is still not very popular in the industry, because fuzzing tools often require very deep tuning for each target program. The advantage of ForAllSecure, he explained, is that a tool like Mayhem has been carefully designed, and the adaptability that comes with it can help Cloudflare promote fuzzing really efficiently and routinely. Haynes also emphasized that symbolic execution is better at finding more complex vulnerabilities, but has traditionally been used mostly in research labs.

Ruoyu Wang, a professor at Arizona State University, hopes the emergence of Mayhem will be the beginning of a future of high-level computer security automation. But he also acknowledged that vulnerability identification is still a tough job requiring full collaboration between humans and robots.

In Wang’s view, Mayhem proved that automation solutions can make a difference in the security field, but existing automatic vulnerability finding tools are still limited in the field of complex Internet services or software packages. Even the most powerful existing software is far from being intelligent enough to truly understand the intent and function of a program the way people do. But on the other hand, Mayhem’s ability to perform multiple tests quickly is something that humans don’t have. “Automated solutions can indeed find a lot of security vulnerabilities that cannot be found by humans alone,” Wang said.

Wang himself was part of the Mechanical Phish team, the safety team that won third place in the 2016 DARPA Safety Championship (the same competition that Mayhem was born in). Now, he is working on a new initiative within the research institute CHESS to develop more powerful bug-finding software. According to the plan, the software will solve difficult problems that existing machines cannot handle with the assistance of humans. Wang concludes, “Currently, state-of-the-art automation technology may still encounter obstacles at any time. New systems should be able to recognize the existence of such obstacles and consult humans at any time.” Although Mayhem can find vulnerabilities autonomously, it will be developed in the future. Among other things, it should be part of the overall security team—not everything.

The Links:   2DI100A-120   LQ038Q5DR01