Programmers obfuscate their code to defeat manual or automated analysis. Obfuscations are often used to hide malicious behavior.
In particular, malicious programs employ obfuscations of stack-based instructions, such as call and return instructions, to
prevent an analyzer from determining which system functions it calls. Instead of using these instructions directly, a combination
of other instructions, such as PUSH and POP, are used to achieve the same semantics. This paper presents an abstract interpretation
based analysis to detect obfuscation of stack instructions. The approach combines Reps and Balakrishnan’s value set analysis
(VSA) and Lakhotia and Kumar’s Abstract Stack Graph, to create an analyzer that can track stack manipulations where the stack
pointer may be saved and restored in memory or registers. The analysis technique may be used to determine obfuscated calls
made by a program, an important first step in detecting malicious behavior.