In a recent attempt to move away from Google analytics I wanted to be able to run the AWS command line tool from inside of Netlify's build process. Except it turns out it wasn't so easy to get AWS's environment variables in Netlify because Netlify have some undocumented reserved environment variables:
The AWS command line tool will only read your credentials from two very defined locations:
- From carefully named environment variables
- From a credentials file store at
The arguments to the tool are very limited, and don't offer options to pass in the credentials. This is understandable as it would the expose secret tokens in the history of the shell (or shell logs).
The second constraint is that, in my case, I had to use the AWS command line tool. If I were using Node.js or Python or Ruby and including the AWS library as a dependency, then I could name the environment values as I please. However I was using
aws from Bash so I was constrained by the two options I mention earlier for authorisation.
I tried many different ways to get this to work and failed many (many) times. I think I can see one other way to solve this problem, but the solution that follows worked for me.
Step 1: alternatively named env values
In Netlify's deploy/environment panel I add the following three environment values:
Step 2: during build, generate credentials
My project uses a Bash script during the build phase. So the first thing I do is to generate a credentials file substituting the environment values in:
cat > ./aws <<EOL [netlify] region = $ENV_DEFAULT_REGION aws_secret_access_key = $ENV_SECRET_ACCESS_KEY aws_access_key_id = $ENV_ACCESS_KEY_ID EOL
This uses a here document to save multiple lines to a file which also substitutes my custom AWS environment values.
Since Netlify's build contents is private, no one has access to read my secret values. I can also test this worked (or understand what failed) by download the build which is tucked away in Netlify's UI:
Step 3: change credentials source
Tucked away inside of AWS's environment values that it reads, is a value that defines where the credentials is read. Why this isn't an argument on the command line, I've no idea.
AWS_SHARED_CREDENTIALS_FILE=./aws aws --profile netlify …
Now the correct credentials are loaded into the
aws tool and I'm able to run the commands I need. In my case, I was calling
aws s3 sync, but that's for another blog post.