How to retrieve remote FTP backup file list via API (password hidden)

A

amoozesh98com

Verified User
Joined
Mar 8, 2015
Messages
25
Hey everyone,

I'm developing a WHMCS module for our DirectAdmin servers and I've hit a wall. The goal is to let our users see what backup files are available on the remote FTP server (the one configured in Admin Backup/Reseller Backup) so they can request a specific restore date via a support ticket.

Everything works fine for local backups – I can get the settings via CMD_API_ADMIN_BACKUP?json=yes and even list the files. But for remote FTP backups, the API returns all the FTP connection details except the actual password. It just shows XXXXXXXXXX or an empty string for ftp_password. I've tried:

  • CMD_API_ADMIN_BACKUP (raw & JSON) – password masked.
  • CMD_API_ADMIN_BACKUP_MODIFY?cron=2 – same.
  • CMD_API_USER_BACKUP and CMD_API_SITE_BACKUP – either empty password or "You do not own that domain".
  • Logging in with a session cookie and parsing the HTML of the backup page – the password field is still XXXXXXXXXX and obviously the browser just shows dots.
  • Task queue, cron jobs API – no password there either.
  • Even trying to read the config files via the file editor API (CMD_API_FILE_EDITOR) just redirects to the login page.
I understand this is a security measure – you shouldn't be able to pull plaintext passwords via the API. But is there any official way to list the contents of the remote FTP backup directory without knowing the password? Something like an API call that uses the stored credentials internally and returns the file list? Or maybe a hook/script that runs on the server and can be triggered via API?

I'm running the latest Evolution skin, and the backup cron is working perfectly (I can see the backup progress and files in the DA panel). I just need a way to expose that file list to my module so users can pick a backup date for restore.

Any pointers would be greatly appreciated. If there's truly no way, I'll just have to ask the admin to manually enter the FTP password into a custom field, but that's not ideal.

Thanks in advance!
 
Well, it's not very smart to have an api return passwords.

How about just registering the backups locally using a web hook like all_backups_post.sh? Use it to update e.g. a backups.json in the user directory. If you need a users backups meta data, just retrieve that file using the api.
 
I need to provide our WHMCS users with a list of available backup files stored on a remote FTP server (configured via Admin Backup in DirectAdmin), including file names, creation dates, and sizes. The goal is to let users pick a specific backup date when requesting a restore via support ticket.

The problem: DirectAdmin API (CMD_API_ADMIN_BACKUP?json=yes) masks the FTP password (shows XXXXXXXXXX), so I can't connect to the FTP server to list files.

I understand that exposing plaintext passwords via API is a security risk. So I'm looking for a secure, recommended approach:

  1. Can I use all_backups_post.sh hook (which receives ftp_password, ftp_ip, ftp_path, etc. as environment variables) to generate a metadata file (e.g., backups.json) in /home/admin/ after each backup, containing file names, sizes, and timestamps?
  2. Then read that JSON file via CMD_API_EXEC or CMD_API_FILE_EDITOR from WHMCS?
  3. Is there a built-in API endpoint that returns the list of files already stored on the remote FTP, using the stored credentials internally without exposing them?
I'm running DirectAdmin v1.689+ with Evolution skin. The backup cron is working perfectly – I just need to expose the file list to my module securely.

Any guidance on the best practice for this scenario would be greatly appreciated.
 
1 + 2 yes.
3. somewhat... not an api endpoint (i think) but you can make that list easily.

DA doesn't need to export the password as it's already available in that hook. It runs after alle backups have run and you can do whatever you want there:

You can e.g. do something like this to get a list of files from the same backup server:

Code: 
#!/bin/bash

output=/home/admin/backups.json
tmp=$(mktemp)

# use a mini ftp script to get the remote directory listing
lftp -u "$ftp_username","$ftp_password" \
    -p "${ftp_port:-21}" "$ftp_ip" <<EOF >"$tmp"
set cmd:fail-exit yes
set ftp:ssl-force no
cd "$ftp_path"
cls -l --time-style=long-iso
bye
EOF

# make a function to format that list in any way you want
my_own_generate_json_from_listing "$tmp" >"${output}.new" && mv "${output}.new" "$output"

rm -f "$tmp"

next, have whmcs retrieve it, of upload the file yourself to the whmcs.

Edit: this is a very basic example. Don't put passwords on the lftp commandline as it is visible in the processlist. Use a lftp config file.
 
I'm using the `all_backups_post.sh` hook to generate backup metadata after each admin backup completes. My goal is to maintain one JSON file per backup type (`ftp_backups.json` and `local_backups.json`) that keeps track of all backup sets, where each unique cron schedule + path gets its own entry that gets replaced each time that specific cron runs.

Current Setup:
- Hook: `/usr/local/directadmin/scripts/custom/all_backups_post.sh`
- The hook receives all environment variables (success, where, owner, ftp_, local_, minute, hour, dayofmonth, month, dayofweek, append_to_path, select*, etc.)
- For FTP backups, I use `curl` to list files and get sizes from the remote FTP server
- For local backups, I use `stat` to read files from disk

What I want to achieve:

A single JSON file structure like this:

```json
{
"type": "ftp",
"created_at": "2026-07-22 10:00:00",
"updated_at": "2026-07-22 16:40:00",
"remote_host": "87.236.210.144",
"backups": {
"0_5_*_*_3___/Wednesday": {
"cron_key": "0_5_*_*_3",
"path": "/Wednesday",
"date": "2026-07-22 05:00:00",
"timestamp": 1751529600,
"users": {
"user1": {"file": "user.admin.user1.tar.zst", "size": "22.52 KB", "size_bytes": 23064},
"user2": {"file": "user.admin.user2.tar.zst", "size": "17.17 KB", "size_bytes": 17582}
}
},
"0_5_*_*_5___/Friday": {
"cron_key": "0_5_*_*_5",
"path": "/Friday",
"date": "2026-07-25 05:00:00",
"timestamp": 1751788800,
"users": {
"user1": {"file": "user.admin.user1.tar.zst", "size": "22.60 KB", "size_bytes": 23150}
}
},
"manual___/custom_backup": {
"cron_key": "manual",
"path": "/custom_backup",
"date": "2026-07-22 14:30:00",
"timestamp": 1751529000,
"users": { ... }
}
}
}
```

Logic required:
1. Unique key = `cron_schedule + path` (e.g., `0_5_*_*_3___/Wednesday` for cron `0 5 * * 3` with path `/Wednesday`, or `manual___/Friday` for manual backups)
2. When a backup runs: **replace only the entry with the same key**, keep all other entries untouched
3. For **cron backups**: key = `minute_hour_dom_month_dow` (from DA environment variables)
4. For **manual backups**: key = `manual` (so re-running manual backup to same path replaces previous manual entry)
5. Must support all `append_to_path` options: `nothing`, `dayofweek`, `dayofmonth`, `weekofmonth`, `month`, `date`, `custom`

Problem:
I keep running into issues parsing the existing JSON file in pure bash. I need to:
- Find and remove the old entry for the current backup set key
- Keep all other entries intact
- Add/update the new entry
- Produce valid JSON with proper commas

Question:
1. Is there a better approach for this? Maybe using a different hook or storing data differently?
2. Should I use `jq` (is it available on all DA servers)?
3. Is there a way to use `CMD_API_ADMIN_BACKUP` or another API to read the cron configuration so I can name entries more intelligently?
4. Any example scripts that handle JSON merging in `all_backups_post.sh`?

Constraints:
- No external tools that aren't available by default on CentOS/AlmaLinux
- Must work with both local and FTP backups
- Must handle all `append_to_path` patterns
- Cron variables: `minute`, `hour`, `dayofmonth`, `month`, `dayofweek` are passed when `when=cron`
- Manual backups have `when=now`
 
Last edited:
What I want to achieve overall

I need to handle all backup scenarios - both scheduled (cron) and manual - including all `append_to_path` options:

- Nothing
- Day of Week (e.g., /Wednesday)
- Day of Month (e.g., /22)
- Week of Month (e.g., /week-4)
- Month (e.g., /Jul)
- Full Date (e.g., /2026-07-22)
- Custom

And both backup trigger methods:
- Now (manual)
- Cron Schedule (with minute, hour, day of month, month, day of week)

The core problem:

In DirectAdmin, when a scheduled or manual backup runs to the same path, it replaces the old backup files on the server (local or FTP). Old backup files get overwritten and are gone.

So if I keep a metadata JSON file that stores information about backups, and a backup gets replaced on the server, my metadata file must also replace that entry - not keep accumulating stale data for backups that no longer exist.

What I need the hook to do:

Build a metadata file (`ftp_backups.json` and `local_backups.json`) where:

1. Each unique cron schedule + path combination gets its own entry
2. When the same cron+path runs again, that entry gets replaced (because the actual backup files on the server got replaced)
3. Entries for other cron schedules or paths remain untouched
4. Manual backups to the same path also replace the previous manual entry for that path

This way, the metadata always reflects what actually exists on the backup storage - no stale data, no orphaned entries for backups that have been overwritten and no longer exist.

The challenge:

I'm struggling with pure bash JSON manipulation - finding the old entry, removing it, keeping everything else intact, and inserting the new entry with proper commas and valid JSON structure.


Thank you so much for your help so far. I'm extremely grateful and I hope you'll continue to help me solve this challenge. Thanks again!
 
Last edited:
Well, I have no idea in what format the whmcs site wants the info but you're free to use any format you want, I guess. Also, Chatgpt/claude can build this for you in 10 minutes.
 
I believe that you are an outstanding individual. Your registration date on the forum since 2007 is truly impressive to me, and it's clear that you are an extremely professional person. I truly believe this and I'm deeply grateful to you for supporting me up to this moment and this point in developing this feature.

Before I can use the JSON file inside WHMCS, I need to be able to generate it properly. Artificial intelligence like ChatGPT Pro or any of these AI models do not have the accuracy, thinking power, and experience that you or a human has.

I first need to be able to generate that JSON file with a standard structure using a script inside DirectAdmin, and then I can write a reader for it to be read and interpreted inside WHMCS.

I'm currently having problems on the DirectAdmin side.

Inside DirectAdmin, you cannot access the backup files on FTP via the API. And if we want to have access, we would need to scan the path, which takes time to get a response due to scanning directories. If the number of users in the backup process is large, scanning and retrieving the names of each individual backup file would be a time-consuming process in the WHMCS response.

Therefore, your suggestion to use metadata to have information about the created backups is absolutely excellent.

But the issue is that in DirectAdmin, there are various types of backup scheduling:

  • Nothing
  • Day of Week (e.g., /Wednesday)
  • Day of Month (e.g., /22)
  • Week of Month (e.g., /week-4)
  • Month (e.g., /Jul)
  • Full Date (e.g., /2026-07-22)
  • Custom
For example, in the Cron Schedule section:
Minute: 0
Hour: 5
Day of Month: *
Month: *
Day of Week: *

There are also custom configurations.

My goal is that if, for example, I had set up a schedule in DirectAdmin for FTP backups on 2 days of the week, days 3 and 5, meaning Wednesday and Friday, backups run twice a week.

We need the backup process to save the FTP backup metadata in such a way that the information for both days of the current week is recorded. Next week, when Wednesday's backup runs, last week's Wednesday information should be removed from the metadata file and the new information for that Wednesday should be updated in the FTP backup metadata file. Then when we get to Friday, last week's Friday metadata should be removed from the file and the fresh backup information for that Friday should be placed in the file.

Because DirectAdmin replaces the backups, we must always keep our metadata file up to date with the new information.

And it should truly show the information for backups that actually exist, whether on FTP or local.

We need to be so precise in generating the metadata that each time we run any of the following scheduled or immediate backup types, the local and FTP backup metadata files are updated in such a way that the old information is replaced with the new information, and nothing is accidentally deleted:

  • Nothing
  • Day of Week (e.g., /Wednesday)
  • Day of Month (e.g., /22)
  • Week of Month (e.g., /week-4)
  • Month (e.g., /Jul)
  • Full Date (e.g., /2026-07-22)
  • Custom
For example, in the Cron Schedule section:
Minute: 0
Hour: 5
Day of Month: *
Month: *
Day of Week: *

Meaning, for example, when a scheduled FTP backup for Wednesday runs, the FTP backup information for Wednesday that was taken last week should be replaced with this new information - not that it deletes the file and removes the backup information for other days too!

We shouldn't be deleting anything, we should only be replacing.

I'm not sure if I was able to convey what I mean or not.
Again, I'm extremely grateful that you're reading my long message and providing any ideas or help. Thank you so much.


And lastly, this is not a ten-minute job. It definitely requires the experience of an experienced human like you, my friend. I'm extremely, infinitely grateful to you. Thank you so much.
 
I'm not recommended to use the admin backup, FTP credential stored on those config.

If you really want to do:
* create plugins
* push API endpoint inside plugins like "/CMD_PLUGINS_ADMIN/custom_api/api/status.raw"
* run test locally by : curl "$(da api-url)/CMD_PLUGINS_ADMIN/custom_api/api/status.raw"

Or workaround version ( Not recommended ):
1 goto "/evo/admin-backup-and-restore/restore"
2 switch to FTP ( that should already saved the credential from the previously create backup/restore ) so you don't need to edit any input.
* ftp_password will look like "XXXXXXXXXX"
3 open browser development tools and switch to network tab,
4 click "RELOAD FILES" and capture request url, body payload / response from "CMD_ADMIN_BACKUP?json=yes"

try testing in the locally
Code: 
curl "$(da api-url)/CMD_ADMIN_BACKUP?json=yes" \
  -H 'accept: application/json' \
  -H 'content-type: application/json' \
  -H 'x-json: yes' \
  --data-raw '<BODY_PAYLOAD req>'


Response should have {files} list array
 
I did what you suggested, but there's an issue with this approach — it doesn't return the file dates, file sizes, and some other important metadata.

The safest and most logical approach would have been to use the metadata method, which our friend here pointed out and guided us on. However, writing a complete and flawless script for that requires enough experience and expertise.

So while your workaround works to some extent, it's missing critical information that we actually need.
 
Last edited:
You must log in or register to reply here.
Back
Top