It's unlikely anyone is spoofing your IP address. The concept is very misunderstood, and here's why: If someone else spoofs your computer's IP address then your server will respond back to your computer, and not to the person who's spoofing your address.
There's no possible way to work around this that I know of or can conceive of.
Chances are it's something at your IP# that you don't know about. If it is a spoof it's strictly a DOS attack; the person can't ever use the attack to get into your server.
We had a client (a network engineer) who appeared to have this problem. Because he knew it was unlikely he put a bit of effort into the search and finally found a system he'd forgotten about, on his network, doing an attempted email login. He shut that off, and the problem disappeared.
Jeff