Cloud Files Container Bulk Deletion Script written in BASH

So, we had a lot of customers asking for ways to delete all of their cloud files in a single container, instead of having to do manually. This is possible using the bulk delete function defined in the rackspace docs. Find below the steps required to do this.

Step 1: Make an auth.json file (for simplicity)

{
    "auth": {
        "RAX-KSKEY:apiKeyCredentials": {
            "username": "mycloudusername",
            "apiKey": "mycloudapikey"
        }
    }
}

It’s quite simple and nothing intimidating.

For step 2 I’m using an application called jq, to install it do


wget https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64
mv jq-linux64 /bin/jq
alias jq = '/bin/jq'

Now you can use jq at the commandline.

Step 2: Set variable called $TOKEN that can store the api token password output, the nice thing is there is no token stored in the script so its kind of secure

TOKEN=`curl https://identity.api.rackspacecloud.com/v2.0/tokens -X POST -d @auth.json -H "Content-type: application/json" | jq .access.token.id | sed 's/"//g'`
echo $TOKEN

Step 3: Set a variable for the container name

# Container to Upload to
CONTAINER=meh2

Step 4: Populate a List of all the files in the $CONTAINER variable, in this case ‘meh2’.

# Populate File List
echo "Populating File List"
curl -X GET https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567/meh2 -H "X-Auth-Token: $TOKEN" > filelist.txt

Step 5: Add container name to the file listing by rewriting the output file filelist.txt to a deletelist.txt

sed -e "s/^/\/$CONTAINER\//" <  filelist.txt > deletelist.txt

Step 6: Bulk Delete Files thru API

echo "Deleting Files.."
curl -i -v -XDELETE -H"x-auth-token: $TOKEN" https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567\?bulk-delete -T ./deletelist.txt

Step 7: Confirm the deletion success

# Confirm Deleted
echo "Confirming Deleted in $CONTAINER.."
curl -i -X GET https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567/meh2 -H "X-Auth-Token: $TOKEN"

The completed script looks like this:


 Mass Delete Container

# Get Token

TOKEN=`curl https://identity.api.rackspacecloud.com/v2.0/tokens -X POST -d @auth.json -H "Content-type: application/json" | jq .access.token.id | sed 's/"//g'`
echo $TOKEN

# Container to Upload to
CONTAINER=meh2


# Populate File List
echo "Populating File List"
curl -X GET https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567/meh2 -H "X-Auth-Token: $TOKEN" > filelist.txt


# Add Container Prefix
echo "Adding Container Prefix.."
sed -e "s/^/\/$CONTAINER\//" <  filelist.txt > deletelist.txt


# Delete Files
echo "Deleting Files.."
curl -i -v -XDELETE -H"x-auth-token: $TOKEN" https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567\?bulk-delete -T ./deletelist.txt

# Confirm Deleted
echo "Confirming Deleted in $CONTAINER.."
curl -i -X GET https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567/meh2 -H "X-Auth-Token: $TOKEN"

Pretty simple!

Running it..

* About to connect() to storage101.lon3.clouddrive.com port 443 (#0)
* Trying 2a00:1a48:7900::100...
* Connected to storage101.lon3.clouddrive.com (2a00:1a48:7900::100) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* subject: CN=storage101.lon3.clouddrive.com
* start date: May 18 00:00:00 2015 GMT
* expire date: Nov 17 23:59:59 2016 GMT
* common name: storage101.lon3.clouddrive.com
* issuer: CN=thawte DV SSL CA - G2,OU=Domain Validated SSL,O="thawte, Inc.",C=US
> DELETE /v1/MossoCloudFS_10045567?bulk-delete HTTP/1.1
> User-Agent: curl/7.29.0
> Host: storage101.lon3.clouddrive.com
> Accept: */*
> x-auth-token: AAA7uz-F91SDsaMOCKTOKEN-gOLeB5bbffh8GBGwAPl9F313Pcy4Xg_zP8jtgZolMOudXhsZh-nh9xjBbOfVssaSx_shBMqkxIEEgW1zt8xESJbZLIsvBTNzfVBlTitbUS4RarUOiXEw
> Content-Length: 515
> Expect: 100-continue
>
< HTTP/1.1 100 Continue HTTP/1.1 100 Continue * We are completely uploaded and fine < HTTP/1.1 200 OK HTTP/1.1 200 OK < Content-Type: text/plain Content-Type: text/plain < X-Trans-Id: tx010194ea9a104443b89bb-00161f7f1dlon3 X-Trans-Id: tx010194ea9a104443b89bb-001611f7f1dlon3 < Date: Thu, 15 Oct 2015 10:25:35 GMT Date: Thu, 15 Oct 2015 10:25:35 GMT < Transfer-Encoding: chunked Transfer-Encoding: chunked < Number Deleted: 44 Number Not Found: 0 Response Body: Response Status: 200 OK Errors: * Connection #0 to host storage101.lon3.clouddrive.com left intact

Leave a Reply

Your email address will not be published. Required fields are marked *