By looking into it a little further, I found that this error has something to do with freetype/config/ftheader.h not being found.
In file included from /usr/local/directadmin/custombuild/php-5.3.28/ext/gd/gd.c:82:
/usr/local/include/ft2build.h:56:38: error: freetype/config/ftheader.h: No such...