Results 1 to 8 of 8

Thread: CLI backup strategy for incremental backup

  1. #1
    Join Date
    Aug 2006
    Location
    LT, EU
    Posts
    7,756

    CLI backup strategy for incremental backup

    Even if tools like ClusterLogics and R1Soft integrate with DirectAdmin, and incremental backup solutions like https://www.modmaster.pro/ exist, if you'd like to use free incremental backup utility (while it gets integrated into DirectAdmin), here is a short description of the idea:
    DirectAdmin has "Admin Backup/Transfer" feature, which could create you a backup without any data in /home. This should take minimal space on the system, as main data (emails and website data) is stored in home. We could use any utility (borg, restic, riff-backup) to backup whole /home folder.

    The steps would be:
    1) Go to "Admin Backup/Transfer" feature, "Schedule" a backup to be ran as often as you'd like to (once a day might be enough), de-select "Domains Directory" and "E-mail data", place your backups to /home/admin/admin_backups for example.
    2) Setup restic (supports many backup storage types) or borg (SSH required on the other end). Listing steps for borg, but things would be very similar for restic too.
    a) install borg, commands for CentOS:
    Code:
    yum -y install epel-release
    yum -y install borgbackup
    Debian:
    Code:
    apt install borgbackup
    b) initialize a local OR remote repository:
    local (using /backups, but you can set any directory you want):
    Code:
    borg init --encryption=none /backups
    remote (using borgbackup SSH user on remote server having privileges to /backups/hostname):
    Code:
    REPOSITORY=borgbackup@YOUR_SERVER_IP:/backups/`hostname -f`
    I'd suggest adding ssh keys from root to that remote server, so that it could connect without a password.

    c) create /usr/local/directadmin/scripts/custom/all_backups_post.sh script, which would automatically sync with borg backup server after every backup done, chmod the script to 700, content:
    Code:
    #!/bin/sh
    REPOSITORY=borgbackup@YOUR_SERVER_IP:/backups/`hostname -f`
    
    
    # Backup all of /home and /var/www except a few
    # excluded directories
    borg create -v --stats                          \
        $REPOSITORY::'{hostname}-{now:%Y-%m-%d_%H:%M}'    \
        /home                    \
        /var/www/html                    \
        /etc                    \
        /usr/local/directadmin > /tmp/borg-stat.tmp 2>&1
    
    
    if [ "$?" -le 1 ]; then
    borg prune -v $REPOSITORY --prefix '{hostname}-' \
        --keep-daily=7 --keep-weekly=4 --keep-monthly=6
    else
        date >> /tmp/borg-stat.tmp
            mail -s "backup failed on server `hostname -f`" your@email.com < /tmp/borg-stat.tmp
    fi
    
    
    # Use the `prune` subcommand to maintain 7 daily, 4 weekly and 6 monthly
    # archives of THIS machine. The '{hostname}-' prefix is very important to
    # limit prune's operation to this machine's archives and not apply to
    # other machine's archives also.
    On any failure it should mail you (your@email.com in script) about problems with it. Retention can be also specified/changed in the script.

    This article is just to show alternative (free) ways to setup and incremental backup, until it's fully integrated into DirectAdmin.
    Last edited by smtalk; 07-07-2019 at 11:06 AM.
    Martynas Bendorius
    MB Martynas IT. Professional server management company. Official DirectAdmin, CloudLinux, LiteSpeed and Comodo partners.

  2. #2
    Join Date
    Jul 2019
    Posts
    7
    hey smtalk, for centos 7, it will be this command

    yum -y install epel-release

  3. #3
    Join Date
    Aug 2006
    Location
    LT, EU
    Posts
    7,756
    Quote Originally Posted by tan87 View Post
    hey smtalk, for centos 7, it will be this command

    yum -y install epel-release
    Sorry, it's been a typo, fixed now. Thanks!
    Martynas Bendorius
    MB Martynas IT. Professional server management company. Official DirectAdmin, CloudLinux, LiteSpeed and Comodo partners.

  4. #4
    Join Date
    Jul 2019
    Posts
    7
    I'd suggest adding ssh keys from root to that remote server, so that it could connect without a password.
    To further hardening this, i suggest to use the append-only mode: https://borgbackup.readthedocs.io/en...pend-only-mode
    This will it make impossible to delete the backups if somehow the DirectAdmin server get hacked.

  5. #5
    Join Date
    Jul 2017
    Location
    Murfreesboro
    Posts
    192
    I am trying restic. Seems similar with more Backup choices..
    Thanks,
    Brent Dacus
    Just a regular guy from Tennessee

    CentOS 7 | DA | CB 2.0 | MariaDB 10.4 | PHP 7.2

  6. #6
    Join Date
    Jul 2017
    Location
    Murfreesboro
    Posts
    192
    Quote Originally Posted by smtalk View Post
    Even if tools like ClusterLogics and R1Soft integrate with DirectAdmin, and incremental backup solutions like https://www.modmaster.pro/ exist, if you'd like to use free incremental backup utility (while it gets integrated into DirectAdmin), here is a short description of the idea:
    DirectAdmin has "Admin Backup/Transfer" feature, which could create you a backup without any data in /home. This should take minimal space on the system, as main data (emails and website data) is stored in home. We could use any utility (borg, restic, riff-backup) to backup whole /home folder.

    The steps would be:
    1) Go to "Admin Backup/Transfer" feature, "Schedule" a backup to be ran as often as you'd like to (once a day might be enough), de-select "Domains Directory" and "E-mail data", place your backups to /home/admin/admin_backups for example.
    2) Setup restic (supports many backup storage types) or borg (SSH required on the other end). Listing steps for borg, but things would be very similar for restic too.
    a) install borg, commands for CentOS:
    Code:
    yum -y install epel-release
    yum -y install borgbackup
    Debian:
    Code:
    apt install borgbackup
    b) initialize a local OR remote repository:
    local (using /backups, but you can set any directory you want):
    Code:
    borg init --encryption=none /backups
    remote (using borgbackup SSH user on remote server having privileges to /backups/hostname):
    Code:
    REPOSITORY=borgbackup@YOUR_SERVER_IP:/backups/`hostname -f`
    I'd suggest adding ssh keys from root to that remote server, so that it could connect without a password.

    c) create /usr/local/directadmin/scripts/custom/all_backups_post.sh script, which would automatically sync with borg backup server after every backup done, chmod the script to 700, content:
    Code:
    #!/bin/sh
    REPOSITORY=borgbackup@YOUR_SERVER_IP:/backups/`hostname -f`
    
    
    # Backup all of /home and /var/www except a few
    # excluded directories
    borg create -v --stats                          \
        $REPOSITORY::'{hostname}-{now:%Y-%m-%d_%H:%M}'    \
        /home                    \
        /var/www/html                    \
        /etc                    \
        /usr/local/directadmin > /tmp/borg-stat.tmp 2>&1
    
    
    if [ "$?" -le 1 ]; then
    borg prune -v $REPOSITORY --prefix '{hostname}-' \
        --keep-daily=7 --keep-weekly=4 --keep-monthly=6
    else
        date >> /tmp/borg-stat.tmp
            mail -s "backup failed on server `hostname -f`" your@email.com < /tmp/borg-stat.tmp
    fi
    
    
    # Use the `prune` subcommand to maintain 7 daily, 4 weekly and 6 monthly
    # archives of THIS machine. The '{hostname}-' prefix is very important to
    # limit prune's operation to this machine's archives and not apply to
    # other machine's archives also.
    On any failure it should mail you (your@email.com in script) about problems with it. Retention can be also specified/changed in the script.

    This article is just to show alternative (free) ways to setup and incremental backup, until it's fully integrated into DirectAdmin.
    The restic one is really cool... I hope you all i thinking of using something with AWS support and not just remote server backups.
    Thanks,
    Brent Dacus
    Just a regular guy from Tennessee

    CentOS 7 | DA | CB 2.0 | MariaDB 10.4 | PHP 7.2

  7. #7
    Join Date
    Aug 2007
    Posts
    19
    What we need - its not only incremental backup on admin side, but possibility to restore from such backup on user side.
    Dou you have such plans in roadmap?
    Russian translations of Directadmin
    Evolution skin: https://github.com/tuthost/directadmin-evolution-ru
    Enhanced skin: https://github.com/tuthost/directadmin-enhanced-ru

  8. #8
    Join Date
    Jun 2019
    Posts
    23
    @smtalk you copy line for the script instead of correct remote repository initiate command ,

    Code:
    borg init --encryption=none borgbackup@YOUR_SERVER_IP:/backups/`hostname -f`


    I'll not copy/paste you the usual extract command that bloggers post, but and interesting way for to make restorations easy

    Strategy for restoration until better tools are added



    Uses bor mount option* + mc (midnight commander)
    Run

    Code:
    yum install mc
    Code:
    mkdir /rbackup
    Code:
    borg list borgbackup@YOUR_SERVER_IP:/backups/`hostname -f`
    You will get result/s similar to this:
    server-name-2019-07-14_16:37 Sun, 2019-07-14 16:37:49 [veverve.....




    Code:
    borg mount borgbackup@YOUR_SERVER_IP:/backups/`hostname -f`::server-name-2019-07-14_16:37 /rbackup

    then you can start mc and just browse the files in the /rbackup

    Code:
    mc
    (pic is generic added to give idea how mc look to people that never saw it or the dos nc )

    https://upload.wikimedia.org/wikiped...untu_11.04.png

    and copy back what is needed (single file / files / folders or account backup files that contain the db)

    after copying what is needed un mount it

    Code:
    borg umount /rbackup

    *require FUSE

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •