Reseller API problem on Spam Filter

_rik_

Verified User
Joined
Sep 25, 2019
Messages
43
Location
England
I've made a script to add rules to the Spam Filter automatically so I don't have to waste weeks to update many accounts by hand.
It's a bash script that uses the reseller user/pass impersonating the user accounts.
It works.
The problem is not adding, is deleting.
The same rules added by my scipt cannot be deleted by the same script (and no error).

I've tried different strategies: by value, by index, all at once, one by one... nothing to do.

This is the specific part of the script that manages the deletion by value
Code:
# 2. Delete each current filter
            CURRENT=$(curl -s -k -u "$RESELLER|$u:$RESELLERPASS" \
                "$DA_HOST/CMD_API_EMAIL_FILTER?domain=$DOMAIN&api=yes" |
                perl -MURI::Escape -ne 'print uri_unescape($_)')

            # Extract lines with both type and value
            grep -oP '[0-9]+=type=[^&]+&value=[^&]+' <<< "$CURRENT" | while IFS='=' read -r idx rest; do
                type=$(echo "$rest" | sed -n 's/.*type=\([^&]*\).*/\1/p')
                value=$(echo "$rest" | sed -n 's/.*value=\([^&]*\).*/\1/p')
                echo "Deleting $type=$value"
                curl -s -k -u "$RESELLER|$u:$RESELLERPASS" \
                    -d "action=delete" \
                    -d "domain=$DOMAIN" \
                    -d "type=$type" \
                    -d "value=$value" \
                    "$DA_HOST/CMD_API_EMAIL_FILTER"  # > /dev/null
            done

It returns:
Deleting domain=domain1.com
Deleting domain=test.it
Deleting domain=shhht.net
and the API returns
error=0&text=Filter%28s%29%20removed&details=
for every deletion. So all OK.

It looks like it works but actually does nothing. There is no error.

Any idea why I can't delete the rules the same script can add without problems? It's a bug?
 
Thanks for the useful reply.

I've managed to make 1 deletion using the ID (so I'm now sure it's not a problem of permissions), but the big problem now is how do I get the real IDs because the API exposes only the position not the ID (at least I can't find it). So this code only deletes the older rule in the list, not all of them.

Code:
# 2. Delete each current filter (using real filter IDs)
            CURRENT=$(curl -s -k -u "$RESELLER|$u:$RESELLERPASS" \
                "$DA_HOST/CMD_API_EMAIL_FILTER?domain=$DOMAIN&api=yes" |
                perl -MURI::Escape -ne 'print uri_unescape($_)')

            # Extract real filter IDs (the part before '=')
            FILTER_IDS=($(grep -oP '^[0-9]+(?==)' <<< "$CURRENT"))

            if [[ ${#FILTER_IDS[@]} -gt 0 ]]; then
                echo "Deleting filters by ID: ${FILTER_IDS[*]}"
                POSTDATA="action=delete&domain=$DOMAIN"

                i=0
                for id in "${FILTER_IDS[@]}"; do
                    POSTDATA="${POSTDATA}&select${i}=${id}"
                    ((i++))
                done

                RESPONSE=$(curl -s -k -u "$RESELLER|$u:$RESELLERPASS" \
                    -d "$POSTDATA" \
                    "$DA_HOST/CMD_API_EMAIL_FILTER")

                echo "DELETE RESPONSE: $RESPONSE"
            else
                echo "No filters to delete for $DOMAIN"
            fi
Deleting filters by ID: 0
DELETE RESPONSE: error=0&text=Filter%28s%29%20removed&details=


Basically, without knowing the ID, the only thing I can do (not very clean) is to do this

Code:
# 2. Delete all filters by looping until empty
while true; do
    CURRENT=$(curl -s -k -u "$RESELLER|$u:$RESELLERPASS" \
        "$DA_HOST/CMD_API_EMAIL_FILTER?domain=$DOMAIN&api=yes" |
        perl -MURI::Escape -ne 'print uri_unescape($_)')

    # Extract current filter IDs (just 0,1,2.. from response)
    FILTER_IDS=($(grep -oP '^[0-9]+(?==)' <<< "$CURRENT"))

    if [[ ${#FILTER_IDS[@]} -eq 0 ]]; then
        echo "No more filters to delete for $DOMAIN"
        break
    fi

    echo "Deleting filters by ID: ${FILTER_IDS[*]}"
    for id in "${FILTER_IDS[@]}"; do
        RESPONSE=$(curl -s -k -u "$RESELLER|$u:$RESELLERPASS" \
            -d "action=delete" \
            -d "domain=$DOMAIN" \
            -d "select0=$id" \
            "$DA_HOST/CMD_API_EMAIL_FILTER")
        echo "  Deleted $id -> $RESPONSE"
    done
done

This works because
CMD_API_EMAIL_FILTER re-numbers filters after every deletion (0,1,2…).
So it always deletes select0=0, then re-fetch, then delete the next 0, etc.
Loop stops once no filters remain.

That's not very nice, lot's of posts, but it works and I since can't find a way to remove all the rules with a single post... I'd say this is... better than nothing.
 
I've checked the newer API (I'm with version 1.684) but there isn't anything of use for this problem.
So the above code it's the best I can do at the moment... that is
Fetch current filters.
Extract all indexes (0,1,2,...).
Send a selectN=<index> request for each one individually.
Re-fetch and repeat until none are left.
 
You can use:

Code:
select0=1
select1=5
select2=10
...
selectN=25

for removing multiple lines at once.

Use API request to:

Code:
/CMD_API_EMAIL_FILTER?domain=domain.com&json=yes&redirect=yes

for JSON output of existing filtering rules.
 
I've already tried that but it only deletes the top filter, not the others. The catch is that the API doesn't returns IDs, it returns positions and they are not OK for such deletion call. That’s why previously, even when I passed select0=0&select1=1&select2=2, only the top filter was deleted (DA re-numbers filters after each deletion even in the same call).
Uhm... maybe I could try to pass 0 multiple times, like
select0=0&select1=0&select2=0
That could be an hybrid approch.
 
The best way is to see how it works in a browser using webinterface of DirectAdmin. I have:

JSON:
{
    "select0":19,
    "select1":20,
    "select2":21,
    "domain":"domain.com",
    "json":"yes",
    "action":"delete"
}
when I delete multiple filters.

and a response:

JSON:
{
    "result": "",
    "success": "Filter(s) removed"
}
 
PS: on second thought, it won't work because the “re-number after each deletion” behavior happens between API calls, not inside a single POST. So I'm stuck with multiple calls.
 
Code:
{
    "select0":19,
    "select1":20,
    "select2":21,
    "domain":"domain.com",
    "json":"yes",
    "action":"delete"
}
only works because you have the IDs. I don't have the IDs and the API doesn't seem to have any call to returns the IDs. It only returns the position and that is not OK for deletion.
 
OK, I have:


JSON:
{
    "filters": 
    {
        "0": 
        {
            "type": "word",
            "value": "test111"
        },
        "1": 
        {
            "type": "word",
            "value": "test222"
        },
        "2": 
        {
            "type": "word",
            "value": "test333"
        },
        "3": 
        {
            "type": "word",
            "value": "test444"
        },
        "4": 
        {
            "type": "word",
            "value": "test555"
        }
    },
    "high_score": "15",
    "high_score_block": "no",
    "where": "inbox"
}


Then I call

Bash:
[root@server ~]# curl -X POST 'https://domain.com:2222/CMD_API_EMAIL_FILTER?action=delete&domain=domain%2Ecom&json=yes&redirect=yes&select0=1&select1=3'
{
        "result": "",
        "success": "Filter(s) removed"
}
[root@server ~]#

to remove the 1st and 3rd rules, and I get:

JSON:
{
    "adult": "OFF",
    "filters": 
    {
        "0": 
        {
            "type": "word",
            "value": "test111"
        },
        "1": 
        {
            "type": "word",
            "value": "test333"
        },
        "2": 
        {
            "type": "word",
            "value": "test555"
        }
    },
    "high_score": "15",
    "high_score_block": "no",
    "where": "inbox"
}


What do I do wrong?
 
I got it wrong about positions and IDs... understood that (and with json is better) it really works now! :)
Here is the new bash code


Code:
# --- DELETE ALL CURRENT FILTERS ---
            while :; do
                CURRENT=$(curl -s -k -u "$RESELLER|$u:$RESELLERPASS" \
                    "$DA_HOST/CMD_API_EMAIL_FILTER?domain=$DOMAIN&json=yes")

                # Extract filter indexes (0,1,2...) using jq
                IDS=$(jq -r '.filters | keys[]' <<< "$CURRENT" 2> /dev/null)

                if [[ -z "$IDS" ]]; then
                    echo "    No more filters for $DOMAIN"
                    break
                fi

                # Build POST with all current indexes
                POSTDATA="action=delete&domain=$DOMAIN"
                i=0
                for id in $IDS; do
                    POSTDATA="$POSTDATA&select$i=$id"
                    ((i++))
                done

                echo "    Deleting filters: $IDS"
                RESPONSE=$(curl -s -k -u "$RESELLER|$u:$RESELLERPASS" \
                    -d "$POSTDATA" \
                    "$DA_HOST/CMD_API_EMAIL_FILTER")

                echo "      RESPONSE: $RESPONSE"
            done
 
Back
Top