Debugging buffer overflows

When debugging a complex parser, you have to deal with buffer overflows. Consider a parser that reads 1-2 characters past the end of buffer when some specific input is provided. The bytes after the buffer are usually readable, but if they are not, the program crashes. It is hard to catch the bug, because it occurres only with some specific memory layout.

In such cases, you need to make the bug occur always. You program will crash more often, but that's good because it helps you to find the root of the problem.

Here is a debugging version of malloc that allocates the buffer at the very end of memory page, so reading one extra character leads to access violation error:

VOID* DbgMalloc(SIZE_T size) {
    SYSTEM_INFO sys_info;
    GetSystemInfo(&sys_info);

    INT_PTR page = sys_info.dwPageSize;
    assert((page & (page - 1)) == 0); // page size is the power of two
    SIZE_T rounded_size = (size + page - 1) & (-page); // round up to the page boundary

    BYTE* start = (BYTE*)VirtualAlloc(NULL, rounded_size + page, MEM_COMMIT, PAGE_READWRITE);
    DWORD old_protect;
    BOOL res = VirtualProtect(start + rounded_size, page, PAGE_NOACCESS, &old_protect);
    assert(res); UNREFERENCED_PARAMETER(res);
    return start + (rounded_size - size);
}

Now you can allocate memory with this function when running automatic tests. If the test coverage is close to 100%, you will able to catch all buffer overflow errors.

Download the source code (10 KB)