Uhm... not allowing the backups to use ram (os determines mostly how this is manged) would probably bring your server down pretty fast as it wouldn't be able to cache (millions?) of files for the backups. It would also increase the IO with A LOT, which you would notice pretty fast if the sites need IO to get te pages. Or am I wrong?
But the 'cache' ram is the first to get freed when the os thinks there is better use for it. Why don't you get a cheap vps and have that rsync your server? That will give you all the ram you have (minus a few gb).
On the other hand... if you can use things like memcached, redis etc... you could probably use that to keep the memory occupied so it won't get assigned to cache.