How-to: cPanel to DA migration

Hi
Check that you have increased your username limit at DA_URL:2222/admin/settings/server at the bottom
then delete the /home/admin/admin_backups/user.gdresell.ilmxjit57bta.tar.zst file and again try to restore just the cp-move file.

if this still does not work contact DA support.
 
As we're getting more and more questions on how to use https://forum.directadmin.com/showthread.php?t=52792&p=280613#post280613, I've decided to write this article to make it clear :)

See the 2nd post if you'd just like to try it for a single or couple accounts.

FAQ for new customers:
http://forum.directadmin.com/showthread.php?t=58062&p=296460#post296460

---
Known issues:
* cPanel packages owned by root are not stored in any backups, thus they need to be converted manually using a script.
* Mailman mailing lists do not get converted to majordomo mailing lists
* Please make sure it's you who created cpmove-user.tar.gz backup for security purposes, because at this moment we fully trust the content inside, thus available on admin level only. This might change later.
* DirectAdmin format for mysql usernames/databases is always username_database and username_user. cPanel has the same format, but due to max username/database length (https://forums.cpanel.net/threads/username-length-limits.630671/), is sometime shortens the username part in from of it. cPanel allows myusername_ to have a DB username with myuserna_. DA DB names/usernames always start with myusername_, changing this would create a mess, for example, you could have myuserna user owning myuserna_db1 and myusername user owning myserna_db2. The script converts username to be username in such cases, so, for some accounts you might need to change database username/name in wp-config.php or other files CMS uses. This should be rare though. Script output in such cases:
Code:
WARNING! us_wordpress cannot be owned by user, renaming database user to user_wordpress
* JetBackup format doesn't include cpmove-user directory inside, so, for now these need to be converted manually, like (would store converted backup in /home/admin/converted_user_backup):
Code:
jetbackup=yes /usr/local/directadmin/scripts/cpanel_to_da/cpanel_to_da.sh /home/admin/user_backups/cpmove-USERNAME.tar.gz /home/admin/converted_user_backup
chown -R admin. /home/admin/converted_user_backup

Other things to take into consideration:
* It's recommended to leave all the cpmove-user.tar.gz files on the system after restore. If there are bugs, or something goes wrong, it'd still be possible to recover that data.
* Like any software - there might be bugs there, if you notice any - please let us know, and we'll try fixing them as soon as possible.
* DirectAdmin supports a different feature set than cPanel. For example, DirectAdmin supports nginx/openlitespeed, MySQL8, rspamd etc., but it has no support for PostgreSQL or Ruby. So, if you have any sites using them - they'd need to be transferred manually.
* Max username length is 16 characters for MySQL users by default, and 10 for system user. Max username length can be increased in /usr/local/directadmin/conf/directadmin.conf. You can find max length of your username in cPanel easily:
Code:
ls /var/cpanel/users | awk '{print length, $0}' | sort -nr | head -n1

To increase it in DA to 16, for example, you can simply run:
Code:
/usr/local/directadmin/directadmin set max_username_length 16 restart
* To make sure your MySQL username/database name is fine, I'd suggest upgrading your MySQL/MariaDB on the server (https://www.directadmin.com/features.php?id=2294), MySQL 5.7+ or MariaDB 10.x is recommended.
* I'd suggest trying it with 1 account first, most preferably - the one with the longest database name/username (guide for this in 2nd post) to make sure you have the nameservers you need and that you don't hit any limitations :)
* If you have user.tar.gz instead of cpmove-user.tar.gz, it will not work! pkacct forms cpmove-users.tar.gz. If you have a list of user.tar.gz files, and want to rename them all in current directory:
Code:
for i in `ls *.tar.gz | grep '^[A-Za-z0-9]*.tar.gz'`; do { mv -vf ${i} cpmove-${i}; }; done
* If you have backup-...._user.tar.gz instead of cpmove-user.tar.gz, it will not work! pkacct forms cpmove-users.tar.gz. If you have a list of backup-...._user.tar.gz files, and want to rename them all in current directory:
Code:
for i in `ls backup-[0-9]*.*.tar.gz`; do { USERNAME=`echo $i | cut -d_ -f3 | cut -d'.' -f1`; mv -v $i cpmove-${USERNAME}.tar.gz; }; done
* If you don't see the user in "Move users between resellers" list, please apply: https://help.directadmin.com/item.php?id=251
* If you did not restore resellers first, all users were assigned to "admin". If you want to re-assign them to correct resellers after restore:
Code:
cd /home/admin/all_backups
for i in `ls user*.tar.gz`; do { RESELLER=`echo $i | cut -d. -f2`; USER=`echo $i | cut -d'.' -f3`; /usr/local/directadmin/scripts/move_user_to_reseller.sh $USER admin $RESELLER; }; done
echo "action=cache&value=showallusers" >> /usr/local/directadmin/data/task.queue
/usr/local/directadmin/dataskq d
* If you want to sync all packages from cPanel to DirectAdmin, including root packages, to admin user, you could simply do (change your_cpanel_server.com to a real domain name, 22 to SSH port if it's custom):
Code:
wget -O /usr/local/directadmin/scripts/cpanel_to_da/copy_root_packages.sh http://files1.directadmin.com/services/all/cpanel-conversion/copy_root_packages.sh
chmod +x /usr/local/directadmin/scripts/cpanel_to_da/copy_root_packages.sh
/usr/local/directadmin/scripts/cpanel_to_da/copy_root_packages.sh your_cpanel_server.com 22
---

MIGRATION STEPS:
Steps to transfer ALL the accounts from cPanel server to DirectAdmin:
cPanel server:
1) Create a backup of all cPanel accounts on cPanel server and place them to /home/all_backups (make sure you have enough of space for them, if not, please read the suggestions at the end of the article):
Code:
mkdir -p /home/all_backups
for user in `ls /var/cpanel/users/`; do { /scripts/pkgacct ${user} /home/all_backups; }; done
2) Transfer /home/all_backups to DirectAdmin server:
Code:
rsync -avt --delete /home/all_backups/ root@your_directadmin_server.com:/home/admin/all_backups/

We're done with cPanel server now. Let's connect to DirectAdmin server now.

DirectAdmin server:
1) Make sure your DirectAdmin version is at least 1.57.4 (as of time of writing, pre-release: https://help.directadmin.com/item.php?id=408), make sure backups at /home/admin/all_backups are owned by "admin" and that you have latest version of convertor script
Code:
cd /usr/local/directadmin/custombuild
./build update
./build cpanel_to_da
chown -R admin. /home/admin/all_backups
2) We finally have DirectAdmin backups in /home/admin/all_backups :) Now we can simply go to DirectAdmin "Admin Backup/Transfer" section, check "Backup/Restore Settings" (defaults should be fine) in Admin level and restore all backups from "/home/admin/all_backups" directory.

That's it!

-----
If you have enough of space, I'd suggest leaving /home/admin/all_backups still there on DA server for a couple of weeks/months, if there is something missing in DA - you'd always have it in that backup.

If your cPanel server doesn't have enough of space - it's easy transfer accounts one-by one directly to the home folder on a new server. The most convenient way to do this is to generate SSH keys on cPanel server using:
Code:
ssh-keygen

Then you'll have your public key placed in /root/.ssh/id_rsa.pub. Just copy the content of it to your DirectAdmin server, file /root/.ssh/authorized_keys (if /root/.ssh directory does not exist - create it). This will make it possible to connect to DA server without any password (from your cPanel server). Now instead of steps 1) and 2) on cPanel server, just do this:
Code:
for user in `ls /var/cpanel/users/`; do { /scripts/pkgacct ${user} /home/all_backups; rsync -avt /home/all_backups/cpmove-${user}.tar.gz root@your_directadmin_server.com:/home/admin/all_backups/cpmove-${user}.tar.gz; rm -f /home/all_backups/cpmove-${user}.tar.gz ; }; done
Your username is included in the name of the backup file, and the digits that follow it indicate the date and time that the backup was made by cPanel.Alternately, users who have root access can generate a full backup using the command prompt by typing the following commands:
/scripts/pkgacct USERNAME (Your Username)

In the /home directory, you will find a newly created archive file with the name cpmove-USERNAME.tar.gz.
The next step is to upload the backup files to the DirectAdmin panel, and they should be placed in the directsay/home/admin/cpanel_backups directory.
Then Navigate to the login page for your DirectAdmin admin account.
Further Browse to the area labeled Admin Tools, then select Admin Backup/Transfer from the menu. Simply click the button labeled "RESTORE."
Input the location of the backup file here. Click the RESTORE button after selecting the backup file to use.
 
I decided to switch to DirectAdmin a while ago, and I'm confident that I made the right decision. Thank you very much for the information.
:)
 
the Cpanel import script have a big problem.

We tried to migrate 70 accounts, 45 failed with a message saying "database is locked", wich on other thread someone from DA i guess, said it was a problem creating the backup on the Cpanel server.

But if we login to cpanel server and with the same packgacct script create the 70 backups, none of them fails

And is not something about server load, or the amount of account, we tried now with just 1 account, and same error database is locked.
 
I've created a new Migration tool to help our resellers move from WHM to DirectAdmin, but this is a free tool for anyone to use, for any Host/servers as long as the host has an FTP service. Basically it automates the backup and transfer process and will also rename the backup files from backup-xxxx.tar.gx to cpmove-xxxx.tar.gz

The end result is the DirectAdmin reseller account will be left with a directory full of all the cpmove files which the reseller can then Restore easily from within DirectAdmin themselves. Basically it just automates everything for the reseller except the last step, but that last step is quick anyway as DirectAdmin automates the restores. So resellers can now do migrations themselves without the need of root or admin intervention/access.

Here is the link: https://deasoft.com/tools/cpanel-directadmin-automated-migration-tool.php
 
Last edited:
I've created a new Migration tool to help our resellers move from WHM to DirectAdmin, but this is a free tool for anyone to use, for any Host/servers as long as the host has an FTP service. Basically it automates the backup and transfer process and will also rename the backup files from backup-xxxx.tar.gx to cpmove-xxxx.tar.gz

The end result is the DirectAdmin reseller account will be left with a directory full of all the cpmove files which the reseller can then Restore easily from within DirectAdmin themselves. Basically it just automates everything for the reseller except the last step, but that last step is quick anyway as DirectAdmin automates the restores. So resellers can now do migrations themselves without the need of root or admin intervention/access.

Here is the link: https://deasoft.com/tools/cpanel-directadmin-automated-migration-tool.php
i would never enter login credentials of Cpanel, FTP or DA to anywebsite, and would recommend anyone to never do that.

If you want to release a tool for public use, release the code and everyone can check it and use it.

just my opinion, it's a big risk.
 
Yeah I understand, what you say is reasonable, but what you suggest is a fair bit more work that will take some time :) - The other option is to reset your passwords immediately after using it, but even so I understand you wouldn't know if it did something malicious before completing its work.

I can state plainly as the person who coded this I have no interest at all in peoples login details or doing something nefarious. The purpose was to initially make life easier for myself and my own clients and then I thought - why don't I make it publicly available and gain some good will and exposure. But yeah thinking about it more, it's probably too much of a security issue for most.

I could release the code, I will think about that. Problem with that is it will no doubt be taken and exploited by others for their own gain...... What I might end up doing is putting it in my DAreseller software.

Anyway thanks for your feedback.

Update - I added a FAQ to answer peoples questions.
 
Last edited:
I have a problem with the migration tool.

When I have a email account with the same name as system user in cpanel the script tries to merge the mail boxes as seen in the logs here:

WARNING! [email protected] matches system account name, merging mailbox with system user's one.

On restore the mailbox is empty at the direct admin end. It hasn't imported the [email protected] email.

On CPanel [email protected] has about 250MB of data. On Directadmin [email protected] ends up with 0 bytes of data in the mailbox. All other mailboxes transfer fine.

If i rename these accounts in cpanel its going to change the name of the databases and I will have to manually fix wordpress etc for about 60 sites.
 
When I have a email account with the same name as system user in cpanel
You shouldn't use names that are used for email as system user. It is better to give every system user a name that wouldn't be used for mail.

Also you should check if there isn't a MBOX or MDBOX now at the system user maildir as sometimes cpanel mbox/mdbox convert will not work and you will not see mail untill you convert the mbox or mdbox to maildir. You can easy do it.

But i would change the system user name on your da server and import cpanel again.
 
the Cpanel import script have a big problem.

We tried to migrate 70 accounts, 45 failed with a message saying "database is locked", wich on other
thread someone from DA i guess, said it was a problem creating the backup on the Cpanel server.

But if we login to cpanel server and with the same packgacct script create the 70 backups, none of them fails
And is not something about server load, or the amount of account, we tried now with just 1 account, and same error database is locked.
when using the same script directly on the CPanel server, the backups are created without any issues. This suggests that the problem might be related to the environment or configuration where the import script is being executed. It's worth investigating potential differences in the environments or any specific settings that could be causing the database lock errors during migration.
 
If there is no support included with your license - please PM me any cpmove backup file which had this issue (you may create a new one, without any data, if you wish). I'd like to check why FTP accounts didn't get migrated there.

Thanks, after some figuring, I was able to create a ticket in the end. The mistake was trying to create the ticket with our ID + the license ID, instead of our ID + our pass.

If you have access, ticket #31750 - the x4 cpmove tgz's are on the server and the ticket has admin details to access it.

Was this ever resolved?

I bring this up, because I'm seeing the same behavior now. Sub FTP users on cPanel are not being copied over to DirectAdmin.

Perhaps the most damning thing is that the directories for these sub FTP users are not being copied over. It's one thing to miss the FTP accounts themselves (i.e. not being able to log in to these special FTP accounts) it's another thing to completely miss the contents of the directories these sub FTP accounts have.

For example, if a sub FTP account on cPanel has the home directory: /home/%user/%domain%/ftp1 at the very least I would expect the directory - /home/%user/%domain%/ftp1 to exist on DirectAdmin when the hosting account is migrated over. But at least for me this directory and it's contents all appear to be missing with the migration. Is the migration script not doing a 1:1 copy of the web hosting account's home directory?

I will also add that I'm kind of creating my own wrapping system around the cpanel_to_da.sh script, so it's possible all of this is missing due to something that I am doing (although I don't know what that would be). But I did find this mention in the thread from @jonathanPcolo which seems to mirror what I am seeing. I'm not seeing any further discussion regarding this issue - so that's why I'm asking now.
 
Ha! Turns out I was kind of on to this in March of last year:


The issue here stems from the fact that cpanel_to_da.sh is going to use zstd regardless if it's enabled in directadmin.conf as long as /usr/bin/zstd (or more specifically command -v zstd) exists.

I don't have zstd enabled on our servers, but the /usr/bin/zstd binary does exist on the server.

This results in the restore process not properly restoring things from the home.tar.zst in the package backup.

I would propose the attached patch that does a system check to see if zstd is enabled in DirectAdmin before changing the TAR_ENDING variable.

You PROBABLY need the same thing for pigz, although it doesn't appear to cause any disruptions like zstd does.

And while you're at it, you may want to do the same thing for gzip - which I assume is tied to the DirectAdmin configuration option backup_gzip

I know I'm probably an outlier in that I don't normally use compression because the saved disk space doesn't counteract the added CPU time to do the compression.

The cpanel_to_da.sh script is just looking for the existence of these compression binaries without any regard of whether or not they are enabled in the DirectAdmin configuration. And then the restore process is acting on what DirectAdmin configuration compression options are enabled.
 

Attachments

I have been using these methos without big issues for a good amount of time.

today we migrated 3 accounts from Cpanel to DA, but 1 of them fails (tried 3 times already)

-----------------------------------------------
We tried manuall moving the cpmove- file, fails with
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
Failed to extract /home/admin/admin_backups/cpmove-USERACC.tar.gz to /home/admin/admin_backups/USERACC_cpanel_to_convert

Error converting file: /home/admin/admin_backups/cpmove-USERACC.tar.gz: Converting rene021516 (/home/admin/admin_backups/cpmove-USERACC.tar.gz)...

File '/home/admin/admin_backups/cpmove-USERACC.tar.gz' has not been restored

-----------------------------------------------

tried the Cpanel import trough DA GUI, fails with
convert stage, parsing cpmove user config: open as tar archive: archive/tar: invalid tar header

---------------------------------------------

anyone familiar with this?
 
Same problem here. Same error, started today for me.

Did you already open a ticket with DA?
 
Same problem here. Same error, started today for me.

Did you already open a ticket with DA?
Hi,

Yes, on ticket support checked and it seems the issue is with the backup itself (not being able to untar it). it's odd, since we can create the backup on cpanel without issues, and have tried 6 times already.

sadly haven't had time to check it further, but i will try today/tomorrow.
 
I'm able to untar the file on mac, so it's not a corrupt file or anything. Something changed in 1.663, because it was working well previously.
 
Still happening, this time with a different cpmove file, and different destination server :\
 

Attachments

  • Castris404 Not Found0725200928.jpg
    Castris404 Not Found0725200928.jpg
    33.2 KB · Views: 41
Back
Top