Backup to S3

[ArashiInteractive]

I've got an DA server running Ubuntu 20.04. Since there's no direct way to backup DA to S3, I've used a few cron jobs strung together with Amazon's aws cli to facilitate regular backups. This works well in that backup files are successfully transferred; however, doing so consumes effectively all my servers' resources. Usage is so high, that websites on the server time out before loading.

My options seem to be set the backup to run in the middle of the night and hope customers don't mind their sites are non-responsive for the 30-45min it takes to transfer a few hundred gigs to Amazon or look for ways to limit the resources aws sync is allowed to consume. Since I'm not using CloudLinux, would cpulimit be the preferred method to accomplish this?

Or have others come up with a better solution?

 

[turbo2ltr]

I just migrated from cpanel. On cPanel after all the backups are created, it calls an after-backup hook script I wrote that uses S3tools to transfer the backups to S3. My script also manages daily, weekly, and monthly snapshots). It has always worked very well for me and I am in the process of setting it up on DA.

Is the slowness you are experiencing while DA is creating the actual backup or is it during the sync to S3?

 

[ArashiInteractive]

There's a slight slowdown while the backup is running, but the major slow down - to the point where sites on the server become inaccessible is during transfer to S3.

I've since purchased a subscription to JetBackup to handle this for me since I haven't had time to dedicate to the problem. Someone else suggested it might be an IO issue, but again, I haven't had the time to look into it further and simply needed a solution that worked.