That's a lot of email. We had a client with 26GB of email in one folder up until a few days ago; but he said it was a compromised account and had us delete it. That took us a few hours; we had to delete the files one at a time. (I created a script to do exim -Mrm on each email.)
Each email is in a separate file. The index files are all I can think of t speed it up; I don't think you want to keep that many emails in memory.
I'd suggest keeping the email indexes on a separate SSD drive if possible, to see if that gives you the speedup yu need. Note it may not; the problem may very well be the resorces used in making suc a large backup.
Jeff