Measuring similarity of windows applications using static and dynamic birthmarks.

A software birthmark is unique, as certain native characteristics of a program, hence can be used to measure the similarity between programs. In general, a static software birthmark does not need program execution, but is more vulnerable to attacks by semantic-preserving transformations. A dynamic software birthmark is applicable to packed executables, but cannot cover all the possible program paths. In this paper, we propose a novel effective technique to measure the similarity of Microsoft Windows applications using both static and dynamic birthmarks, which are based on the list of system APIs as well as the frequency of system API calls. Because system APIs are located in Windows system directories and act as a bridge between applications and the operating system, our birthmarks are resilient to obfuscations and compiler optimizations. A static birthmark consists of the system API call frequency of a target program, which can be extracted by scanning the executable file. A dynamic birthmark is the frequency of system API function calls, which can be extracted by a binary instrumentation tool during the execution of the program. To evaluate the effectiveness of the proposed technique, we compare various types of Windows applications using both the static and dynamic birthmarks. To demonstrate the robustness, we compare packed executables that were compressed by a binary packing tool. We carry out additional experiments for measuring the similarity between target Windows applications at the source code level and verify the evaluation results. The experimental results show that our birthmarks can effectively measure the similarity between Windows applications, as intended.