Amazon's documentation has a handy full example of making authenticated requests from Python:

# AWS Version 4 signing example

# EC2 API (DescribeRegions)

# See: .html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac 
import requests # pip install requests

# ************* REQUEST VALUES *************
method = 'GET'
service = 'ec2'
host = 'ec2.amazonaws'
region = 'us-east-1'
endpoint = ''
request_parameters = 'Action=DescribeRegions&Version=2013-10-15'

# Key derivation functions. See:
# .html#signature-v4-examples-python
def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = os.environ.get('AWS_ACCESS_KEY_ID')
secret_key = os.environ.get('AWS_SECRET_ACCESS_KEY')
if access_key is None or secret_key is None:
    print 'No access key is available.'
    sys.exit()

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope


# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# .html

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query 
# string (use '/' if no path)
canonical_uri = '/' 

# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters

# Step 4: Create the canonical headers and signed headers. Header names
# and value must be trimmed and lowercase, and sorted in ASCII order.
# Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'x-amz-date:' + amzdate + '\n'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the 
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;x-amz-date'

# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").
payload_hash = hashlib.sha256('').hexdigest()

# Step 7: Combine elements to create create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash


# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (remended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request).hexdigest()


# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()


# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in 
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# The request can include any headers, but MUST include "host", "x-amz-date", 
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'x-amz-date':amzdate, 'Authorization':authorization_header}


# ************* SEND THE REQUEST *************
request_url = endpoint + '?' + canonical_querystring

print '\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++'
print 'Request URL = ' + request_url
r = requests.get(request_url, headers=headers)

print '\nRESPONSE++++++++++++++++++++++++++++++++++++'
print 'Response code: %d\n' % r.status_code
print r.text

It doesn't have one for Node.js, though. So I attempted to translate that whole thing into Node:

var request = require('request');
var Crypto = require("crypto-js");
var strftime = require('strftime');

function sign(key, msg) {
    return Crypto.HmacSHA256(key, msg, {asBytes: true});
}

function getSignatureKey(key, dateStamp, regionName, serviceName) {
    var kDate = sign(dateStamp, 'AWS4' + key);
    var kRegion = sign(regionName, kDate);
    var kService = sign(serviceName, kRegion);
    var kSigning = sign('aws4_request', kService);

    return kSigning;
}

function sha256(str) {
    return Crypto.SHA256(str);
}

// ************* REQUEST VALUES *************
var method = 'GET';
var service = 'ec2';
var host = 'ec2.amazonaws';
var region = 'us-east-1';
var endpoint = '';
var request_parameters = 'Action=DescribeRegions&Version=2013-10-15';

var access_key = process.env.AWS_ACCESS_KEY_ID;
var secret_key = process.env.AWS_SECRET_ACCESS_KEY;

var date = new Date();
var amzdate = strftime('%Y%m%dT%H%M%SZ', date);
var datestamp = strftime('%Y%m%d', date);

// ************* TASK 1: CREATE A CANONICAL REQUEST *************
var canonical_uri = '/';
var canonical_querystring = request_parameters;
var canonical_headers = 'host:' + host + '\n' + 'x-amz-date:' + amzdate + '\n';
var signed_headers = 'host;x-amz-date';

var payload_hash = sha256('');
var canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash;

// ************* TASK 2: CREATE THE STRING TO SIGN*************
var algorithm = 'AWS4-HMAC-SHA256';
var credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request';
var string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  sha256(canonical_request);

// ************* TASK 3: CALCULATE THE SIGNATURE *************
var signing_key = getSignatureKey(secret_key, datestamp, region, service);
var signature = sign(signing_key, string_to_sign);

// ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
var authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature;
var headers = {'x-amz-date':amzdate, 'Authorization':authorization_header, host: host};
var request_url = endpoint + '?' + canonical_querystring;

var options = {
    url: request_url,
    headers: headers
};

request(options, function(err, res) {
    console.log(res.statusCode, res.body);
});

When I set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in my shell to appropriate values from my AWS account and run the Python script, I get back status 200 with a bunch of data. But when I run my Node.js script, I get status code 401 and a body that says AWS was not able to validate the provided access credentials. Where did I go wrong?

Amazon's documentation has a handy full example of making authenticated requests from Python:

# AWS Version 4 signing example

# EC2 API (DescribeRegions)

# See: http://docs.aws.amazon./general/latest/gr/sigv4_signing.html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac 
import requests # pip install requests

# ************* REQUEST VALUES *************
method = 'GET'
service = 'ec2'
host = 'ec2.amazonaws.'
region = 'us-east-1'
endpoint = 'https://ec2.amazonaws.'
request_parameters = 'Action=DescribeRegions&Version=2013-10-15'

# Key derivation functions. See:
# http://docs.aws.amazon./general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = os.environ.get('AWS_ACCESS_KEY_ID')
secret_key = os.environ.get('AWS_SECRET_ACCESS_KEY')
if access_key is None or secret_key is None:
    print 'No access key is available.'
    sys.exit()

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope


# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon./general/latest/gr/sigv4-create-canonical-request.html

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query 
# string (use '/' if no path)
canonical_uri = '/' 

# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters

# Step 4: Create the canonical headers and signed headers. Header names
# and value must be trimmed and lowercase, and sorted in ASCII order.
# Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'x-amz-date:' + amzdate + '\n'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the 
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;x-amz-date'

# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").
payload_hash = hashlib.sha256('').hexdigest()

# Step 7: Combine elements to create create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash


# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (remended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request).hexdigest()


# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()


# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in 
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# The request can include any headers, but MUST include "host", "x-amz-date", 
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'x-amz-date':amzdate, 'Authorization':authorization_header}


# ************* SEND THE REQUEST *************
request_url = endpoint + '?' + canonical_querystring

print '\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++'
print 'Request URL = ' + request_url
r = requests.get(request_url, headers=headers)

print '\nRESPONSE++++++++++++++++++++++++++++++++++++'
print 'Response code: %d\n' % r.status_code
print r.text

It doesn't have one for Node.js, though. So I attempted to translate that whole thing into Node:

var request = require('request');
var Crypto = require("crypto-js");
var strftime = require('strftime');

function sign(key, msg) {
    return Crypto.HmacSHA256(key, msg, {asBytes: true});
}

function getSignatureKey(key, dateStamp, regionName, serviceName) {
    var kDate = sign(dateStamp, 'AWS4' + key);
    var kRegion = sign(regionName, kDate);
    var kService = sign(serviceName, kRegion);
    var kSigning = sign('aws4_request', kService);

    return kSigning;
}

function sha256(str) {
    return Crypto.SHA256(str);
}

// ************* REQUEST VALUES *************
var method = 'GET';
var service = 'ec2';
var host = 'ec2.amazonaws.';
var region = 'us-east-1';
var endpoint = 'https://ec2.amazonaws.';
var request_parameters = 'Action=DescribeRegions&Version=2013-10-15';

var access_key = process.env.AWS_ACCESS_KEY_ID;
var secret_key = process.env.AWS_SECRET_ACCESS_KEY;

var date = new Date();
var amzdate = strftime('%Y%m%dT%H%M%SZ', date);
var datestamp = strftime('%Y%m%d', date);

// ************* TASK 1: CREATE A CANONICAL REQUEST *************
var canonical_uri = '/';
var canonical_querystring = request_parameters;
var canonical_headers = 'host:' + host + '\n' + 'x-amz-date:' + amzdate + '\n';
var signed_headers = 'host;x-amz-date';

var payload_hash = sha256('');
var canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash;

// ************* TASK 2: CREATE THE STRING TO SIGN*************
var algorithm = 'AWS4-HMAC-SHA256';
var credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request';
var string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  sha256(canonical_request);

// ************* TASK 3: CALCULATE THE SIGNATURE *************
var signing_key = getSignatureKey(secret_key, datestamp, region, service);
var signature = sign(signing_key, string_to_sign);

// ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
var authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature;
var headers = {'x-amz-date':amzdate, 'Authorization':authorization_header, host: host};
var request_url = endpoint + '?' + canonical_querystring;

var options = {
    url: request_url,
    headers: headers
};

request(options, function(err, res) {
    console.log(res.statusCode, res.body);
});

When I set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in my shell to appropriate values from my AWS account and run the Python script, I get back status 200 with a bunch of data. But when I run my Node.js script, I get status code 401 and a body that says AWS was not able to validate the provided access credentials. Where did I go wrong?

• javascript
• python
• node.js
• amazon-web-services

• 1 did you take a look at this: aws.amazon./sdk-for-node-js ? Should provide some more info about sending authenticated requests to AWS. Official library: npmjs./package/aws-sdk People have made some smaller libraries for specific services as well. e.g. npmjs./package/s3 – Are Almaas Commented Dec 13, 2015 at 13:25
• Yeah I checked out the official SDK but it just gives you black-box functions you can call to access its supported services and doesn't help you make your own authenticated requests. It doesn't support Amazon Elasticsearch Service, the one I need to hit, so I'm on my own. – The Legend of Dave Commented Dec 13, 2015 at 13:36
• Any success in translating it to node js? – ItayAmza Commented Aug 20, 2016 at 10:16
• I am stuck trying to do the same thing with S3 right now – 1Mojojojo1 Commented Oct 16, 2018 at 18:05
• 1 Another issue with just using the SDK is of course if try to run this in something time sensitive, like a lambda, the warmup could be a matter of added seconds. I have so far found only this little lib, but not yet tested it: github./mhart/aws4 – Lior Kupers Commented Jun 3, 2021 at 13:45

1 Answer 1

Well.. I guess I am 4 years late, but I am replying in case it helps someone else:

There is a sample in AWS documentation here: https://docs.aws.amazon./elasticsearch-service/latest/developerguide/es-request-signing.html#es-request-signing-node

If you have the credentials in advance and you need to set them you can do something like this:

var credentials = new AWS.Credentials({ accessKeyId: 'KIA3...', secretAccessKey: '9wqwAvpht...', sessionToken: null });

For security reasons you don't want to hardcode your accessKey and secret key in your code so you will need to encrypt them, or use ssm or kms :-)