Automated Vulnerability Discovery System Based on Hybrid Execution

At present, program vulnerabilities occur frequently, which seriously threatens the security of cyberspace. Automated vulnerability discovery technology has attracted more and more attention because of its efficiency and universality. In the current field of vulnerability discovery, it is common to combine fuzzing with symbolic execution technology. Symbolic execution technology is often used to solve the input of complex paths and help fuzzing improve the program coverage, so as to detect vulnerabilities better. However, the symbolic execution has no goals in the process of hybrid execution exploration path, it is easy to cause the execution path to deviate from the expected target points. And generating the test case of the corresponding path requires frequently calling the solver for solution, while the current solver has low efficiency and poor solvability for nonlinear operation. In order to solve the above problems, we propose an automated vulnerability discovery system based on hybrid execution. In the system, we propose the symbolic execution guidance algorithm based on dynamic and static combination to guide the symbolic execution to solve the input reaching the target points, so as to avoid exploring in useless paths and avoid consuming a lot of time and computing resources. In addition we hook some nonlinear functions to optimize the nonlinear function constraint solving, so as to improve the hybrid execution efficiency. We have conducted extensive experiments on the RHG 2019 challenge dataset and the RHG 2021 challenge dataset. The experimental results show the effectiveness and scalability of the system.