The only way I found so far to cancel an on-going system backup is to kill the two sysbk processes first and then eventually the tar processes.
If you kill just the tar process, sysbk will continue the backup of the next directory...
sample :
# ps -ef | grep sysbk
root 12059 12052 0 17:45 ? 00:00:00 /bin/sh /usr/local/sysbk/sysbk -q
root 12085 12059 0 17:45 ? 00:00:00 /bin/sh /usr/local/sysbk/sysbk -s
root 17737 32554 0 17:47 pts/0 00:00:00 grep sysbk
# kill 12059 12085
A tar process may still be active.
# ps -ef | grep tar
root 17696 1 0 17:47 ? 00:00:00 /bin/tar -cpz --exclude=*/backups/* --file //backup/09-04-06/custom/home//whatever.tar.gz whatever
root 18042 32554 0 17:47 pts/0 00:00:00 grep tar
# kill 17696
Then remove the backup directory
# cd /backups
# rm -rf 09-04-06
There are no significant temporary files AFAIK.
After killing sysbk you will receive a system message "An error occurred during the backup."
There's always the option of just rebooting the server, if you can afford it.
Good luck,
Jaume.