Is there a way to lock mail accounts for maintenance?

kristian

Verified User
Joined
Nov 4, 2005
Messages
436
Location
Norway
Hi,

Sometimes when we're moving accounts between DA servers, the admin backup/restores takes a long time, because the account contains lots of data. The majority of data is generally either inside domains/ somewhere, or inside imap/. In order to speed up the backup/restore and minimize the downtime, I was thinking about doing the migration in roughly these steps:

1. Rsync domains/ and imap/ to new server
2. Use custom httpd config to put their website in "maintenance mode"
3. Hold mail to the domain on our external MX servers
4. Something to avoid the customer to make changes to mailboxes on the server
5. Move domains/ and imap/ out from the home directory
6. Perform the admin backup
7. Perform the restore on new server
8. Move rsynced domains/ and imap/ into the new home directory
9. Release website from maintenance and unhold mails

This way the time spent in step 6 is minimized, since it doesn't need to back up several gigs of data. Since delivery of new mail is stopped, I won't have issues with incoming mail that might be missing from the backup, but there might be people with email clients out there, making changes to their mailboxes that may or may not be brought over to the new server, depending on the timing. So what I'm after is a way to prevent end users from accessing and making changes to their mailbox (step 4). I know I can change the passwords for all the accounts, but this would probably result in their IP address being blocked by LFD, which might not be desirable either. Any suggestions on how I could achieve this?

(The same risk of course goes for data in domains/, but I think the chance of changes there is smaller than for mail accounts.)
 
Saw your old post about having maintenance mode on account for doing backups. Did you find your solution to lock that user before migration? What about using the DirectAdmin 'suspend user'? With that might it lock everything on that user? Have not tried but I think it is a very important step to put a site on maintenance mode before doing a backup. I have seen many restore failures when doing live site backups. The 'suspended user' feature might be the best maintenance mode in DA with less polite messages. I have not tried that but I think it will stop any changes for that user.
 
I did not find any good solution to this, no. Suspending the user would for sure stop any changes, so it might be the best way to do it, apart from manually locking passwords. It feels wrong though, somehow. I don't know enough of what suspend actually does to consider it one way or the other.
 
I just tested now. When suspended, Exim responds to deliveries with
Code:
451 The domain <readacted> is currently suspended. Try later.

But in our case, we have MX servers in front that we hold mail on anyway (step 3 in the list above), during this migration process.
 
Quite an old post but I'd like to toss my idea into the ring:

In this case, I would actually disable the SMTP server entirely. This has great benefits because an unreachable SMTP server triggers an email to retry later in almost, if not exactly, every case. Even if that SMTP server is your own SMTP server sitting in front of it, your SMTP server is going to see the DA SMTP server as down and hold the email to retry when it comes back up. This should be consistently reproducible and safe for a bare minimum of 24 hours without losing any email. Bonus points, it's the absolute easiest of available choices.
 
In this case, I would actually disable the SMTP server entirely.
You mean the MTA otherwise said Exim to be a bit more clear. Because SMTP is for sending mail, right? But during transfers you want to put incoming transfers on hold mostly.
Which is why we stop the Exim service before doing a transfer.

This works nicely, however, in the last line @MaXi32 talked about "that user". I wouldn't suggest stopping these services if it's only 1 user to be transferred. Maybe there is a nicer solution for that, I don't know.
 
You mean the MTA otherwise said Exim to be a bit more clear. Because SMTP is for sending mail, right? But during transfers you want to put incoming transfers on hold mostly.
Which is why we stop the Exim service before doing a transfer.

This works nicely, however, in the last line @MaXi32 talked about "that user". I wouldn't suggest stopping these services if it's only 1 user to be transferred. Maybe there is a nicer solution for that, I don't know.

Correct, SMTP is for inbound and outbound. Though it can be easy to confuse that as Dovecot acts as the LDA so while the SMTP server receives the email, the IMAP server actually ends up writing the file to the folder after having been passed the email by the SMTP server. So just shut off exim and you'll be unreachable for inbound email as needed.

Though yeah, if it's just one user that does change things. Though frankly, I just do it like this:

1. Rsync user's mail directory to new server.
2. Update MX records.
3. Rsync user's mail directory to new server again.
4. Wait a few hours.
5. Rsync user's mail directory to new server one last time.

Worst case scenario they end up seeing a few emails again that they deleted, which isn't too bad for performing a live migration with a DNS change.
 
What I think I'm going to do, is make use of Dovecot's extra fields, in order to solve 4) in my list.

The extra fields can be added to /etc/virtual/<domain>/passwd at the end, and using the nologin and reason fields, the login will be stopped, even when authentication succeeds. Sadly, I suspect email clients will just throw up the same old "Login failed" message, but not much we can do about that.

As part of my "put domain in maintenance" step, I would then update the /etc/virtual/<domain>/passwd file to something like this:

Code:
# cat /etc/virtual/unigroup.no/passwd
myuser:<password here>:1042:8::/home/username/imap/domain.tld/myuser:/bin/false:userdb_quota_rule=*:bytes=50M nologin=y reason=migration
otheruser:<password here>:1042:8::/home/username/imap/domain.tld/otheruser:/bin/false:userdb_quota_rule=*:bytes=50M nologin=y reason=migration
someuser:<password here>:1042:8::/home/username/imap/domain.tld/someuser:/bin/false:userdb_quota_rule=*:bytes=5M nologin=y reason=migration

This does not apply to system users though. In order to also be able to set nologin=y for them, we'll need to override the default passdb section used for those logins, by using /etc/dovecot/conf/custom_passdb.conf which is included before the default sections. It should contain:

Code:
passdb {
  username_filter = !*@*
  driver = shadow
  result_success=continue-ok
}

passdb {
  username_filter = !*@*
  driver = passwd-file
  args = username_format=%Ln /etc/dovecot/system-user-settings
  skip = unauthenticated
}

Short explanation is that this will authenticate the user normally, then allow another passdb section to run, which will do a lookup in the system-users-settings file (file must exist, but can be empty, or dovecot will complain).

This file must comply with the standard passwd-file format, but only needs username and the extra fields, for example:
Code:
username:*::::::nologin=y reason=migration

I noticed that when an email accounts password is changed in DirectAdmin, these extra fields are removed. It would be really nice if they can be left intact (not just for this purpose, but for others, such as allow_nets that I posted about in https://forum.directadmin.com/threa...s-by-ip-to-force-webmail-usage-and-2fa.65755/ - anyone who can look into if this is easily fixed?). Since system-user-settings is a separate file, it will be left untouched even when their password is changed.

Edit/Update: Just for reference, the response dovecot will send when nologin=y is set, is as follows for IMAP:
Code:
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN] Dovecot DA ready.
. LOGIN username password
. NO [CONTACTADMIN] migration
. logout
* BYE Logging out
. OK Logout completed.

And for POP3:
Code:
+OK Dovecot DA ready.
USER username
+OK
PASS password
-ERR [AUTH] migration
quit
+OK Logging out


Any thoughts about this way of doing it?
 
To avoid extra fields being wiped out on password/limits change, the same approach can be done for virtual users. Currently I have this in /etc/dovecot/conf/custom_passdb.conf:

Code:
# The first two will deal with "system users", i.e. main account, without a '@'
# in the username provided.
passdb {
  username_filter = !*@*
  driver = shadow
  result_success=continue-ok
}

passdb {
  username_filter = !*@*
  driver = passwd-file
  args = username_format=%Ln /home/%Ln/.dovecot.system-user-settings
  skip = unauthenticated
}

# The next two will deal with "virtual users".
passdb {
  username_filter = *@*
  args = username_format=%n /etc/virtual/%d/passwd
  driver = passwd-file
  result_success=continue-ok
}

passdb {
  username_filter = *@*
  driver = passwd-file
  args = username_format=%n /etc/virtual/%d/user-settings
  skip = unauthenticated
}

This will let me set extra fields for any accounts, and they will be left alone. I also changed location of the extra files, so they would (hopefully) be included in the DirectAdmin backup.
 
This will let me set extra fields for any accounts, and they will be left alone. I also changed location of the extra files, so they would (hopefully) be included in the DirectAdmin backup.
Bad news is that /etc/virtual/<domain>/user-settings is not included in the DirectAdmin backup for some reason. I guess the process is a bit more selective as to what to keep. The /home/<user>/.dovecot.system-user-settings is included as expected for any file in the user's home directory.
 
I changed around a bit how I did this, but I think I'm happy with the result.

This is my /etc/dovecot/conf/custom_passdb.conf:
Code:
# Custom passdb sections to allow for per-user extra fields to pass to dovecot.
#
# Why extra fields in additional files? Because the shadow driver does not
# support adding extra fields, and the standard virtual passwd file would wipe
# out extra fields added, when a user gets a new password or a change of limit.
#
# This file is included *before* the default passdb sections in dovecot.conf,
# meaning these ones will be used instead of the defaults.

# NOTE: These extra files will *not* be included in a standard DirectAdmin Admin
# Backup. Use custom hook scripts to deal with the files during backup/restore
# if you need. Check out `user_backup_compress_pre` and
# `user_restore_post_pre_cleanup`.

# The first two will deal with "system users", i.e. main account, without a '@'
# in the username provided.
passdb {
  username_filter = !*@*
  driver = shadow
  result_success=continue-ok
}

passdb {
  username_filter = !*@*
  driver = passwd-file
  args = username_format=%Ln /etc/virtual/system-users-settings/%Ln/user-settings
  skip = unauthenticated
}

# The next two will deal with "virtual users".
passdb {
  username_filter = *@*
  args = username_format=%n /etc/virtual/%d/passwd
  driver = passwd-file
  result_success=continue-ok
}

passdb {
  username_filter = *@*
  driver = passwd-file
  args = username_format=%n /etc/virtual/%d/user-settings
  skip = unauthenticated
}

In order to keep the user-settings files updated, you'll also need these custom scripts:

/usr/local/directadmin/scripts/custom/user_create_post/dovecot_system_user_settings.sh:
Bash:
#!/bin/bash

# https://www.directadmin.com/features.php?id=183
# user_create_post/dovecot_system_user_settings.sh - Runs AFTER the user is
# created.

# Environmental variables we care about:
# username=username
# domain=domain.com

# Create a directory for the user and a file that can contain custom user
# settings (extra fields) read by Dovecot. This requires Dovecot to be
# configured to read them as well.
mkdir -p "/etc/virtual/system-users-settings/${username}"
chmod 0755 "/etc/virtual/system-users-settings/${username}"

# The file we write must comply with the format of the passwd-file as described
# in
# <https://doc.dovecot.org/3.0/configuration_manual/authentication/passwd_file/>
echo "${username}:*::::::" > "/etc/virtual/system-users-settings/${username}/user-settings"
chown -R mail:mail "/etc/virtual/system-users-settings/${username}"
chmod 0644 "/etc/virtual/system-users-settings/${username}/user-settings"

# Also write the system user into the virtual user user-settings file, if the
# DirectAdmin config option system_user_to_virtual_passwd is set to 1.
system_user_to_virtual_passwd=$(/usr/local/directadmin/directadmin config-get system_user_to_virtual_passwd)
if [ ${system_user_to_virtual_passwd} -eq 1 ]; then
    echo "${username}:*::::::" >> "/etc/virtual/${domain}/user-settings"
    chown -R mail:mail "/etc/virtual/${domain}/user-settings"
    chmod 0644 "/etc/virtual/${domain}/user-settings"
fi

/usr/local/directadmin/scripts/custom/user_destroy_post/dovecot_system_user_settings.sh:
Bash:
#!/bin/bash

# https://www.directadmin.com/features.php?id=183
# user_destroy_post/dovecot_system_user_settings.sh - Runs AFTER the user is
# destroyed.

# Environmental variables we care about:
# username=username

# Delete the system user user settings directory in /etc/virtual. Compare the
# path with the canonicalized path to ensure we don't delete something we don't
# want to.
if [ "/etc/virtual/system-users-settings/${username}" == "$(readlink -f "/etc/virtual/system-users-settings/${username}")" ]; then
    rm -rf "/etc/virtual/system-users-settings/${username}"
fi

/usr/local/directadmin/scripts/custom/domain_create_post/dovecot_system_user_settings.sh:
Bash:
#!/bin/bash

# https://www.directadmin.com/features.php?id=183
# domain_create_post/dovecot_system_user_settings.sh - Runs AFTER a domain is
# created.

# Environmental variables we care about:
# username=username
# domain=domain.com

# Write the system user into the virtual user user-settings file, if the
# DirectAdmin config option system_user_to_virtual_passwd is set to 1.
system_user_to_virtual_passwd=$(/usr/local/directadmin/directadmin config-get system_user_to_virtual_passwd)
if [ ${system_user_to_virtual_passwd} -eq 1 ]; then
    echo "${username}:*::::::" >> "/etc/virtual/${domain}/user-settings"
    chown -R mail:mail "/etc/virtual/${domain}/user-settings"
    chmod 0644 "/etc/virtual/${domain}/user-settings"
fi

/usr/local/directadmin/scripts/custom/email_create_post/dovecot_virtual_user_settings.sh:
Bash:
#!/bin/bash

# https://docs.directadmin.com/developer/hooks/email.html#email-create-pre-post-sh

# Add a line for the virtual user being created. This lets us set extra fields
# that are read by Dovecot, if it's configured to do so. The file must comply
# with the format of the passwd-file as described in
# <https://doc.dovecot.org/3.0/configuration_manual/authentication/passwd_file/>
echo "${user}:*::::::" >> "/etc/virtual/${domain}/user-settings"
chown mail:mail "/etc/virtual/${domain}/user-settings"
chmod 0644 "/etc/virtual/${domain}/user-settings"

/usr/local/directadmin/scripts/custom/email_destroy_post/dovecot_virtual_user_settings.sh:
Bash:
#!/bin/bash

# https://docs.directadmin.com/developer/hooks/email.html#email-destroy-pre-post-sh

# Delete the user-settings line for the virtual user being destroyed.
sed -i "/^${user}:/d" "/etc/virtual/${domain}/user-settings"

In order to get the files included in the backup, I did this: https://forum.directadmin.com/threads/need-extra-files-included-in-admin-backups.67570/

You can now use dovecot's extra fields in these custom files for whatever purpose you want, and DirectAdmin won't mess with them. I think. :)
 
Back
Top