Amazon AWS EC2 API URL Signature format with ColdFusion

I was trying to connect to EC2 API this weekend and it left me feeling rather weary. I was not trying to do a heart surgery, just to format a string in a particular way and encode it. No matter what I tried AWS slammed me with the error:

The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details

Amazon explains their Signature Version 2 format here. But seems like bits of helpful information got lost between making that document and me reading it. Let me explain signature creation process step by step.

Here is the URL I'm trying to sign:

(AWS Access credential values I'm displaying here is not valid)

It takes three steps, first create a signature string, encode that string with HMACSHA256 or HMACSHA1 and finally attach the encoded value to the end of the URL with the variable name "Signature"

Here is the final result with signature value and this is what you supose to send to AWS API

Create the signature string:
The signature string should looks something like this

   1: GET
   3: /
   4: AWSAccessKeyId=AKIAJH43UFXANDWWWL6A&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2013-02-05T16%3A14%3A13&Version=2012-04-01
Show/Hide Line Numbers . Full Screen . Plain

There are few rules involved in creating this string and every single detail is vitally important - pay attention.

This string spread across 4 lines and lines should be separated by char 10 (Not by char 13, or char 13 and 10 or \n).

Every variable and value is Case Sensitive. (For example, it is AWSAccessKey, not AWSAccesskey)

(I wrap text in following image, but the text string contains only 4 lines as above text block)

Line 1 : GET or POST

Line 2 : Host name of the endpoint (I'm calling EC2 API, hence I'm using, but different AWS APIs have different endpoints.) This must be lower case.

Line 3 : Absolute path portion of the URI (string between .com and ? in the URI). For example if your endpoint is then this should be /cobranded-ui/actions/start . If you do not have a absolute path component, use / (That is what I am using above since EC2 API do not call into a sub folder)

Line 4 : This is the complete query string. It need special sorting and formatting:

First part of the query string must be your AWSAccessKey (number 4 in the image). Note: there is no "?" front of that.

Rest of the string contains all other attributes sorted alphabetically (number 5 in the image).

Here is the trick that took me two days to figure out. Only RFC 3986 specified reserved characters and space characters in values in the query string (not attributes, only values - but all values) must be percent encoded. That mean you may not able to use general URLEncode() functions in your toolkit.

Look at the number 7 in the image. The original string is 2013-02-05T16:25:50 and final string is 2013-02-05T16%3A14%3A13 . that is percent encoded. "-" did not got encoded since it is not an RFC 3986 reserved character, but since ":" is a reserved character, we used the encoded value - %3A . And this values must be Upper Case.

If you follow all those instructions, now you have a proper signature string ready to be encoded.

You can encode this with HmacSHA256 or HmacSHA1 but you must provide correct encode type you have used in the SignatureMethod attribute in the query string. Leave the encoding to your programming language.

Convert the encoded value into base64 and attach to the end of your URL request with the attribute Signature.

Here are ColdFusion functions I used to create the final url with the signature. This use Java class to do the encryption and I coped that function from here. Pass your Query String to awsurl() function, without attributes Timestamp, SignatureVersion, Version or SignatureMethod. The function will add those attributes, values and do alphabetical sorting and the percent encoding. 

   1: <cffunction name="awsurl" returntype="string" access="public" output="No">
   2:  <cfargument name="Query" type="string" required="true" />
   3:  <cfargument name="AWSAccessKeyId" type="string" required="true" />
   4:  <cfargument name="SecretKey" type="string" required="true" />
   5:  <cfargument name="Host" type="string" default="" />
   6:  <cfargument name="Methord" type="string" default="GET" />
   7:  <cfargument name="URI" type="string" default="/" />
   8:  <cfargument name="SignatureVersion" type="string" default="2" />
   9:  <cfargument name="Version" type="date" default="2012-04-01" />
  10:  <cfargument name="http" type="string" default="https" hint="https|http" />
  12:  <!--- add common values --->
  13:  <cfset local.Time = dateConvert("local2Utc",now())>
  14:  <cfset local.Time = "#DateFormat(local.Time,'yyyy-mm-dd')#T#TimeFormat(local.Time,'HH:mm:ss')#">
  15:  <cfset arguments.Query = "#arguments.Query#&Timestamp=#local.Time#&SignatureVersion=#arguments.SignatureVersion#&Version=#arguments.Version#&SignatureMethod=HmacSHA256">
  17:  <!--- sort QueryString --->
  18:  <cfset arguments.Query = ListToArray(arguments.Query,'&')>
  19:  <cfset ArraySort(arguments.Query,'text')>
  20:  <!--- prepend AccessKeyID --->
  21:  <cfset ArrayPrepend(arguments.Query, "AWSAccessKeyId=#arguments.AWSAccessKeyId#")>
  22: <!--- url encode each values --->
  23:  <cfset local.SortedString = ArrayNew(1)>
  24:  <cfloop from="1" to="#ArrayLen(arguments.Query)#" index="local.i">
  25:  <cfset local.value = arguments.Query[local.i]>
  26:  <cfif listlen(local.value,'=') gt 1>
  27:  <cfset ArrayAppend(local.SortedString,"#listfirst(local.value,'=')#=#reservedEncod(listlast(local.value,'='))#")>
  28:  <cfelse>
  29:  <cfset ArrayAppend(local.SortedString,"#local.value#")>
  30:  </cfif>
  31:  </cfloop>
  32: <!--- create signature string --->
  33:  <cfset local.toEncode = "#arguments.Methord##chr(10)##arguments.Host##chr(10)##arguments.URI##chr(10)##ArrayToList(local.SortedString,'&')#">
  34:  <cfoutput><pre>#local.toEncode#</pre></cfoutput>
  35:  <!--- encode Signature String --->
  36:  <cfset local.Signature = URLEncodedFormat(ToBase64(HMAC_SHA256(local.toEncode,arguments.SecretKey)))>
  38:  <cfset arguments.Query = ArrayToList(arguments.Query,'&')>
  39:  <cfset arguments.Query = "#arguments.http#://#arguments.Host##arguments.URI#?#arguments.Query#&Signature=#local.Signature#">
  40:  <cfreturn arguments.Query>
  41: </cffunction>
  42: <!--- ********************************************************************* --->
  43: <cffunction name="reservedEncod" returntype="string" access="public" output="no">
  44:  <cfargument name="string" type="string" required="true" />
  46:  <cfset local.reserved = "!|##|$|&|'|(|)|*|+|,|/|:|;|=|?|@|[|]| ">
  47:  <cfloop list="#local.reserved#" index="local.i" delimiters="|">
  48:  <cfif find(local.i,arguments.string)>
  49:  <cfset arguments.string = replace(arguments.string,local.i, "%#ucase(FormatBaseN(Asc(local.i),'16'))#", 'all')>
  50:  </cfif>
  51:  </cfloop>
  52:  <cfreturn arguments.string>
  53: </cffunction>
  54: <!--- ********************************************************************* --->
  55: <cffunction name="HMAC_SHA256" returntype="binary" access="public" output="no">
  56:  <cfargument name="signMessage" type="string" required="true" />
  57:  <cfargument name="signKey" type="string" required="true" />
  58:  <cfset local.jMsg = JavaCast("string",arguments.signMessage).getBytes("iso-8859-1") />
  59:  <cfset local.jKey = JavaCast("string",arguments.signKey).getBytes("iso-8859-1") />
  60:  <cfset local.key = createObject("java","javax.crypto.spec.SecretKeySpec") />
  61:  <cfset local.mac = createObject("java","javax.crypto.Mac") />
  62:  <cfset local.key = local.key.init(local.jKey,"HmacSHA256") />
  63:  <cfset local.mac = local.mac.getInstance(local.key.getAlgorithm()) />
  64:  <cfset local.mac.init(local.key) />
  65:  <cfset local.mac.update(local.jMsg) />
  66:  <cfreturn local.mac.doFinal() />
  67: </cffunction>
Show/Hide Line Numbers . Full Screen . Plain

Monday 08 September 2014 12:33 AM
This function is a lifesaver. It worked on the first try. After pounding my head against the same problem, I likely would not have discovered the solution on my own. I used the function with the host This solution is a real contribution to the CF community. Keep up the great work.
Friday 14 March 2014 04:04 PM
Hope this helps you in 2014
Friday 18 October 2013 11:25 AM
This is great and it works on a GET request perfectly, however, when the request is changed to a POST I'm having difficulties getting it to work. I get the error SignatureDoesNotMatch. I'm sure it's something to do with the URL encode or with the way of which the elements are concatenated together, however, I can't seem to pinpoint the issue....
Friday 18 October 2013 10:13 PM
I use POST with SES in production. Check
Did you passed correct header param?
Friday 26 April 2013 07:39 AM
hellow sir i have prob with my aws query string
i will try to get claim code from amazon for amazon gift card on demand using aws payment api:


The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

plz help me..
Friday 26 April 2013 07:40 AM
i am using C# for these
Monday 20 May 2013 11:46 AM
To C