Seminar: High-performance, Cross-platform Vetting of the Closed-source Software Ecosystem

10:30 am
Wednesday April 26th, 2023
Room 1263
Patrick F. Taylor Hall




The root cause of over 90% of cyberattacks is security vulnerabilities. At 25 bugs for every 1,000 lines of code, the discovery of security flaws is vital to mitigating cyberattacks. The go-to method of vulnerability discovery is the developer-derived test case: a developer encodes their understanding of program behavior in a set of test cases, then executes those test cases to verify that the program behaves as expected. Unfortunately, the programmer’s mental model of the program is often incomplete and over-constrained. These limitations cause programmers to miss many security vulnerabilities that stem from seemingly impossible test cases. Coverage-guided mutational fuzz testing (i.e., fuzzing) fills in the gaps in testing that developers leave by being underconstrained, i.e., testing with an “anything is possible” mindset. Being underconstrained will find vulnerabilities that developers miss but requires throwing millions of test cases at the program, as being underconstrained means that most test cases will be uninteresting from a program behavior perspective. This results in test case execution rate as the critical metric of fuzzing effectiveness.
In this talk, I will tell you about my work on increasing the test case execution rate of fuzzing. The central observation of my work is that fuzzers spend over 95% of their time executing a test case that will eventually be discarded as uninteresting. Leveraging this observation, I build a fuzzer that encodes the frontier of test case exploration into the program binary so programs self-report when a test case is interesting. The fuzzer then spends the effort on monitoring the coverage of only the 1/10,000 test cases that prove interesting. I call this Coverage-Guided Tracing (CGT). CGT removes the overhead of monitoring code coverage of every executed test case and improves performance by over 600%. On top of CGT, I add support for the most common code coverage metrics shown to increase fuzzing effectiveness, namely edge coverage and hit count coverage---without sacrificing performance. I will conclude the talk with a look at the current fuzzing work underway in my lab that focuses on increasing the fuzzing's performance on Windows and how to better leverage program source code for higher-performance fuzzing.

Elias Bou-Harb

Matthew Hicks
Virginia Polytechnic Institute