eavesROP: Listening for ROP Payloads in Data Streams

We consider the problem of detecting exploits based on
return-oriented programming. In contrast to previous works we investigate
to which extent we can detect ROP payloads by only analysing
streaming data, i.e., we do not assume any modifications to the target
machine, its kernel or its libraries. Neither do we attempt to execute any
potentially malicious code in order to determine if it is an attack. While
such a scenario has its limitations, we show that using a layered approach
with a filtering mechanism together with the Fast Fourier Transform, it
is possible to detect ROP payloads even in the presence of noise and
assuming that the target system employs ASLR. Our approach, denoted
eavesROP, thus provides a very lightweight and easily deployable mitigation
against certain ROP attacks. It also provides the added merit
of detecting the presence of a brute-force attack on ASLR since library
base addresses are not assumed to be known by eavesROP.