Lecture Notes in Computer Science, 2002, Volume 2513/2002, 25-36, DOI: 10.1007/3-540-36159-6_3

Encoding Function Pointers and Memory Arrangement Checking against Buffer Overflow Attack

Changwoo Pyo and Gyungho Lee

View Related Documents

Abstract

Program counter is the only mechanism for processor to access instruction to execute. Protecting program counter is the fundamental defense for securing computer system. This paper presents a scheme of protecting program counter by encoding function pointers. In the scheme, every function address is encoded by linker. Compiler inserts instructions for decoding function addresses before call instruction. Encoding code pointers, function addresses as well as return addresses in stack frame, provides substantial coverage of protecting program counter. Several suggestions are also made to detect compromised code pointers at run-time without memory space for sensor mechanism. A demo Linux system has been under construction with the proposed scheme. Experimental data shows performance slowdown less than 10% when all return addresses and function addresses are encoded. With a Pentium III processor of 866MHz, the overhead for each function call is on the order of nanoseconds. We plan to migrate parts of our code pointer encoding scheme from linker to dynamic linker, which should improve security and performance.

Keywords  buffer overflow attack - program counter - function pointer encoding - memory arrangement checking

This work was supported in part by NSF (CCR-0113409)

Fulltext Preview

Image of the first page of the fulltext document