Backing up asset files to an S3 bucket with the cli

Hi
I’ve done a bit of searching on this but can’t find any examples of how to do it.

We would like to backup contentful asset files using the cli to an amazon s3 bucket using a Lambda function. I have successfully utilised this tutorial to backup the contentful space and this works well. But this does not deal with the asset files themselves.

Looking at the documentation I see there is an option to download the files

https://www.contentful.com/developers/docs/tutorials/cli/import-and-export/

--download-assets

But I’d like to see an implementation of this in a Lambda script like the one in the above example.

Are there any out and about that I can take a look at?

Many thanks

Hi @mark.lett ,

From what I have seen in the article you mentioned, the only additional parameters you will need to pass is downloadAssets: true in the options. Just be aware that the assets will be downloaded in a folder called images.ctfassets.net that exists in the same directory where you will save your JSON file (so you might want to rename the folder).

Hi Alma,

This works on the cmd line but not in the Lambda script as it has nothing in there to say upload images to s3. When I run it as a test from the Lambda function I can see in the logs that it is downloading the assets. I assume to the tmp folder but then they need to be added to the bucket. So that seems to be the missing piece of the puzzle.

I just wanted to know if there is a tutorial or a script out there that will do this or maybe an amend to the script above to take care of the assets as well?

Hi @mark.lett

The CLI import-export tool is designed to be used mostly by an active user in a terminal. If you want to have an higher degree of automation, you could use it as a module in a NodeJS script, and considering integrating also our Management API (CMA) if needed. Although I’m not an expert in AWS, I would say that your Lambda function should first mount the S3 bucket, and then in the NodeJS script you could pass to the export function a similar parameter as the CLI to define your export folder. That I guess would need to match your S3 bucket path. Hope this recommendation might help :smiley:

In Lambda try and download the assets to /tmp/ folder 500MB allowed by default.

Then use Xmaanager upload the tmp folder to S3 Bucket.