How to back up Digital Ocean Spaces (S3 compatible storage)

date posted: 2020-Jun-07, last edit date: 2020-Jul-03

Useful resources:

• Using high-level (s3) commands with the AWS CLI
• Amazon S3 as a File System?
• How to use AWS CLI with Digital Ocean Spaces?
• Recursive list s3 bucket contents with AWS CLI

STEP 1: In your digital ocean dashboard…

1. go to account
2. API
3. spaces access key
4. generate new key
5. save both the access key, and secret key in a text file

STEP 2: Go to your space settings, and add a new entry in the “CORS Configurations” as follows…

• origin == *
• allowed methods == GET
• allowed headers == Access-Control-Allow-Origin

STEP 3: Follow the below script on your local machine…

# install required packages
$ sudo apt install s3fs
$ sudo apt install awscli

# store credentials in a file 
# note: file location is important, see man pages for more details
$ echo ACCESSKEY:SECRETKEY > ~/.passwd-s3fs
$ cat ~/.passwd-s3fs

# restrict file permissions
$ chmod 600 ~/.passwd-s3fs
$ ll ~/.passwd-s3fs

# create the location where you want to mount the S3 drive
$ mkdir ~/s3-drive

# mount drive
# note: check that the drive is not already mounted first
$ s3fs -o del_cache -o passwd_file=~/.passwd-s3fs -o use_path_request_style -o url=https://<S3 space name>.<region name>.digitaloceanspaces.com/ <folder name> ~/s3-drive

# check that the drive is mounted
$ mount | grep s3

# step into the drive
$ cd ~/s3-drive

# list all available files
$ aws s3 ls s3://<S3 space name>/ --endpoint=https://<region name>.digitaloceanspaces.com --recursive | less

# copy files needed from bucket to local
$ aws s3 cp s3://<S3 space name>/<path to folder that you want to copy>/ /<local path> --endpoint=https://<region name>.digitaloceanspaces.com --recursive

# unmount drive
fusermount -u ~/s3-drive