Contact Support

S3 Authentication

If you wish to publish certain objects so they can be accessed publicly without requiring authentication, this is something easily done via the bucket settings. However, by default for operational access (adding/removing objects, labeling, etc.) or just for read access to private objects, authentication is required.

S3 uses a particularly unique (and somewhat complicated) method of authentication.

If you have used APIs before, you most likely used a token generated for you by the server based on a username/password combination or generated from within your account.
You perhaps may have even used an API that used username/password authentication directly.

In modern S3 implementations, however, instead of using a static token you receive from the service you instead derive your own "token" per-request based on your secret key.

There are two commonly used methods for S3 token derivation: the older AWS(2) scheme and the newer AWS4 scheme. Both perform a signed checksum of information about the request.
It is recommended to use AWS4 whenever possible.

Due to the somewhat complicated nature of generating these per-request authentication tokens, it is recommended to use an existing S3 client SDK/library if possible for your project. However, if this authentication needs to be performed manually it can be performed as outlined below.

You will receive three pieces of information (two of which are used in the authentication process) from either NetFire or your company's approved Cloud Storage administrator(s):

  • A username (in the format of TenantID$UserName). This is not used in S3 authentication directly, but may be visible/available in object/bucket metadata for more human-friendly operations.
  • An S3 Access Key. This is not secret and may be freely shared (and in some cases may be necessary to share to allow e.g. sharing ACLs across peers in your tenant).
  • An S3 Secret Key. As the name implies, this is secret. This must be kept in a safe, secret place.

📘

NetFire will never ask for your secret key!

NetFire staff do not need your S3 Secret Key; please do not provide or offer it and report anyone claiming to be NetFire staff that is asking for your S3 Secret Key.
NetFire staff may, on occasion, ask for the S3 Access Key or username of your account to assist in support cases, but we will never ask for your S3 Secret Key. Treat requests for this as illegitimate and not originating from NetFire staff.

AWS2

🚧

MD5 Deprecation

MD5 checksumming, and thus the Content-MD5 header consequently, is considered obsolete and deprecated. Specifically, the Content-MD5 header was deprecated in RFC 7231 Appendix B. This is part of why AWS4 is recommended over AWS2.

🚧

SHA1 Deprecation

SHA1 checksumming, and thus the HMAC-SHA1 method consequently, is considered obsolete and deprecated. This is part of why AWS4 is recommended over AWS2.

General Format

In order to use AWS2 authentication, the following pieces of information are constructed/gathered and then joined with a newline (\n) between each (even if empty):

  • The HTTP method, capitalized (e.g. GET, PUT, etc.)
  • The MD5 sum of the body content represented in base64 encoding (this should also be present as the Content-MD5 header) (if performing a request where no body is present such as a GET, this will be an empty string)
  • The MIME type (per RFC 9110 § 8.3) of the body content (this should also be present as the Content-Type header) (if performing a request where no body is present such as a GET, this will be an empty string)
  • The timestamp of the request's creation (this must also be provided via either the Date header per RFC 9110 § 6.6.1 or the x-amz-date canonical header below) -- this must conform to the RFC 9110 § 5.6.7 timestamp format
  • Canonical Headers (see below)
  • Canonical Resource (see below)

The combination and concatenation of these may be referred to as the request's data to be signed or, abbreviated, sigdata.

This is then signed with your S3 Secret Key via HMAC-SHA1 to create the signature or, abbreviated, sig (your language's cryptographic libraries or packages/modules/third-party libs should offer a function to perform HMAC-SHA1 signing. If the choice is offered, be sure to select the function/method that signs static data/blobs/blocks, not streams).

The resulting bytes from this are then base64-encoded.

Once fully constructed, signed, and base64-encoded, the authentication/signature token is then added as an Authorization header using the AWS type and prefixed with your S3 Access Key and a colon.

Pseudo-code Example

In pseudo-code, this procedure looks something like this:

# Get the Base64-encoded MD5 sum of the body, if present.
if body is not '':
	body_md5 = BASE64(MD5(body))
	Headers['content-md5'] = body_md5
	content_type = Headers['content-type']
else:
	body_md5 = ''
	content_type = ''

# Build the header list, per the "Canonical Headers" section below.
hdr_filtered = []
c_headers = ''
for hdr_name, hdr_value in Headers:
	hdr_name = LOWER(hdr_name)
	hdr_value = STRIP(REPLACE(hdr_value, '\s+', ' '))
	if HAS_PREFIX(hdr_name, 'x-amz-'):
		ADD(hdr_names, hdr_name + ':' + hdr_value)
hdr_filtered = SORT(hdr_filtered)
c_headers = JOIN('\n', hdr_filtered)

# Create the resource string, per the "Canonical Resource" section below.
c_rsrc = ''
if BucketSpecified:
	c_rsrc += tenant_name + ':' + bucket_name + ''
c_rsrc += URL_UNESCAPE(Request['url_path'])
crsrc_append = []
for param_name, param_value in SORT(Request['url_parameters']):
	if param_name not in SubresourceNames:
		continue
	ADD(crsrc_append, param_name + '=' + param_value)
if LEN(crsrc_append) is not 0:
	c_rsrc += '?' + JOIN('&', crsrc_append)

# Build the "sigdata".
sigdata = JOIN(
		'\n',
		[
			method,
			body_md5,
			content_type,
			TIME_NOW().FORMAT('RFC9110'),
			c_headers,
			c_rsrc
		]
	)

# Base64 the signature of the "sigdata" to get the "sig".
sig = BASE64(HMAC_SHA1(secret_key, sigdata))

# And add the header to the request. Note the space after 'AWS'.
Headers['Authorization'] = 'AWS ' +  access_key + ':' + sig

Canonical Headers

If familiar with S3, you may have used x-amz-* (or X-AMZ-*, etc.) headers before. In order to create the canonical headers format, you must collect all these X-AMZ--prefixed headers, normalize their names to lowercase, and sort them alphanumerically based on the header name.

This list of headers and their values are then combined/condensed, their values trimmed (newlines/tabs replaced with a single space, surrounding whitespace removed), and joined with a newline (\n); e.g. this set of headers/values:

X-AMZ-Meta-Username: bob
X-AMZ-Meta-Username: alice
X-AMZ-Date: Tue, 15 Nov 1994 08:12:31 GMT

is transformed into the following string:

x-amz-date:Tue, 15 Nov 1994 08:12:31 GMT\nx-amz-meta-username:bob,alice

Canonical Resource

While at first glance the canonical resource may appear to be a URL path, that's not quite necessarily true (especially if you're using DNS buckets).

The canonical resource string is constructed by concatenating the below:

  • The canonical request URL path:
    • For path buckets, in a request to https://s3.netfire.com/tenant:bucket/file.txt, the canonical request URL path is /<tenant>:<bucket>/file.txt
    • For DNS buckets, in a request to https://<bucket>.s3.netfire.com/file.txt, the canonical request URL path is again /<tenant>:<bucket>/file.txt (note the prefix of the tenant and bucket at the base of the URL, the tenant ID is still required!)
    • If performing a request that does not directly involve a bucket reference (e.g. a bucket listing), simply append /.
  • The "subresource" URL query parameters (e.g. versionid, lifecycle, etc.), if any, in alphanumeric order of the parameter/subresource name (e.g. ?region=us-east&versionid=1&zone=us-east1)

AWS4

The AWS4 signature is somewhat slightly more complex than AWS2 and performs more operations, and so is slightly slower per-request. It is, however, more secure than AWS2. Notably, it replaces MD5 checksumming of the body with SHA256, and HMAC-SHA1 signing with HMAC-SHA256. Most modern S3 client libraries either prefer or only support AWS4.

General Format

In order to use AWS4 authentication, the following pieces of information are constructed/gathered (in order) and then joined with a newline (\n) between each (even if empty):

This newline-joined string makes up the signing message, a "summary" of the request.

In addition to this, a signing key is derived. This documentation refers to this key as k⁴.

Then the signature, or sig, itself is created by lowercase hex encoding (with zero-padded 2-byte pairs) an HMAC-SHA256 signature of the signing message using k⁴ as the signing key.

An alphanumerically-sorted list of the signed headers (just the header name, not the value) joined with a semi-colon is constructed as sigheaders.

Finally, the Authorization header is added with type AWS4-HMAC-SHA256 and the following:

Credential=<access_key>/<YYYYMMDD>/<zone>/s3/aws4_request,SignedHeaders=<sigheaders>,Signature=<sig>

For example, like such:

Authorization: AWS4-HMAC-SHA256 Credential=C0FF33A7D34DB33FC4F3/20230308/us-east1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=67f94996f270809f4755f62d4720dc1390ad8bb7bca97a7169ea20184dd8b147

Pseudo-code Example

In pseudo-code, this procedure looks something like this:

# Get the hex-encoded SHA256 sum of the body, if present.
# It's a constant if not, so don't waste the cycles.
# This is used in a couple places.
if body is not '':
	body_sha = LOWER(HEX(SHA256(body)))
	Headers['x-amz-sha256'] = body_sha
	content_type = Headers['content-type']
else:
	body_sha = 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'

# Get the time formats needed.
t = TIME_NOW()
date = t.FORMAT('YYYYMMDD')
timestamp = t.FORMAT('RFC3339')

# The scope.
scope = JOIN(
		'/',
		[
			date,
			zone,
			's3',
			'aws4_request'
		]
	)

# Build the canonical URI path.
c_uri_path = URL_ENCODE(Request['url_path'])

# Build the canonical query string.
params_append = []
norm_params = {}
for param_name, param_value in Request['url_parameters']:
	norm_params[URL_ENCODE(param_name)] = URL_ENCODE(param_value)
for param_name, param_value in SORT(norm_params):
  if param_value is not null:
		ADD(params_append, param_name + '=' + param_value)
	else:
		ADD(params_append, param_name + '=')
c_params = JOIN('&', params_append)

# Build the canonical headers and signed headers.
shdrs = []
chdrs = []
for hdr_name, hdr_value in Headers:
	hdr_name = LOWER(hdr_name)
	hdr_value = STRIP(REPLACE(hdr_value, '\s+', ' '))
	if hdr_name in [
		'x-forwarded-for',
		'x-forwarded-proto',
		'x-real-ip',
		'connection',
		'authorization',
	]:
		continue
	ADD(shdrs, hdr_name)
	ADD(chdrs, hdr_name + ':' + hdr_value)
shdrs = SORT(shdrs)
chdrs = SORT(chdrs)
s_hdrs = JOIN(';', shdrs)
c_hdrs = JOIN('\n', chdrs)
# Canonical headers require a trailing newline.
c_hdrs += '\n'

# Build the canonical request.
c_req_raw = JOIN(
		'\n',
		[
			method,
			c_uri_path,
			c_params,
			c_hdrs,
			s_hdrs,
			body_sha
		]
	)
c_req = LOWER(HEX(SHA256(c_req_raw)))

# Now build the signing message.
signmsg = JOIN(
		'\n',
		[
			'AWS4-HMAC-SHA256',
			timestamp,
			scope,
			c_req
		]
	)

# Then derive the signing key (k4).
k0 = 'AWS4' + secret_key
k1 = HMAC_SHA256(k0, date)
k2 = HMAC_SHA256(k1, zone)
k3 = HMAC_SHA256(k2, 's3')
k4 = HMAC_SHA256(k3, 'aws4_request')

# Create the signature.
sig = LOWER(HEX(HMAC_SHA256(k4, signmsg)))

# Add the authorization header.
Headers['Authorization'] = JOIN(
		' ',
		[
			'AWS4-HMAC-SHA256',
			JOIN(
				',',
				[
					'Credential=' + access_key + '/' + scope,
					'SignedHeaders=' + s_hdrs,
					'Signature=' + sig
				]
			)
		]
	)

Signing Message

The signing message acts as a sort of summary of the request that contains various components that should be guaranteed to be authentic by the client.

Scope

The scope serves as a restricting identifier to the specific instance of the object you are referencing. For example, it ensures that the object in us-east1 is the correct object, not a version at us-west1 that may be locked to a different version/revision of the object.

It is constructed by joining the following with a slash (/):

  • The date of the request (in format YYYYMMDD, e.g. 20230308 for March 8, 2023)
    • The date must contain (e.g. be the same day as) the timestamp in the signing message!
  • The zone of the object (e.g. us-east1)
  • The service
    • The service is always s3.
  • The string aws4_request

Thus an example scope would be 20230308/us-east1/s3/aws4_request.

Canonical Request

The canonical request contains key information about the HTTP request itself. Signing it ensures a lack of tampering from any proxies or ISPs/carriers between the client and the API.

Canonical URI Path

The canonical URI path provides a way to derive an unambiguous request URL path that the API server and client can agree upon. It is similar to the canonical resource in AWS2.

This is done by URL-encoding the absolute path of the URL. For example, if you have a file foo<space>bar.jpg at https://bucketname.s3.netfire.com/foo bar.jpg, the canonical URI path is actually:

/foo%20bar.jpg

If you need a quick ASCII-to-hex conversion table, one can be found here. Note that you should use capital letters (e.g. A-F) in URL-encoding for the hex representation. If your library does not offer URL encoding, you must implement it yourself via the logic found in RFC 3986 § 2.4.

Canonical Query String

The canonical query string is constructed in a somewhat similar fashion to the canonical resource's subresources, except that:

  • It contains all URL query parameters
  • Parameter names and values are URL encoded (see above)
    • The parameter name and the parameter value are URL-encoded separately; that is, the equals sign between them should not be encoded
  • If a non-valued parameter is specified (e.g. ?someOption instead of ?someOption=someValue), the parameter should be treated as if it had an empty value (e.g. .g. ?someOption=).

All parameters are first converted to URL-encoded parameter names and URL-encoded parameter values, and then sorted by parameter name alphanumerically.

They are then joined together with &. Unlike the canonical resource, there is no ? prepended.

Canonical Headers

AWS4 canonical headers follow the same formatting rules (lowercase normalization, space trimming on the value, and sorting by header name) as AWS2 canonical headers, with the following additional provisions:

  • The Host header must be included
  • If there is a body (e.g. a PUT), then Content-Type must be included (both in the request and in the canonical headers); see the description of this in AWS2's general format
  • The X-AMZ-SHA256 header must be included (both in the request and in the canonical headers)
    • This is a lowercase hex-encoded SHA256 hash of the body. If no body is provided (e.g. this is a GET request), the SHA256 hash of an empty set of bytes should be used (e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855)
  • All X-AMZ-* headers must be included as canonical headers
  • There is a trailing newline after all normalized headers and their values are joined

Additional headers may be included as long as they are in the request itself as well.

🚧

Some headers must not be included as canonical headers!

The following headers must not be included as request headers, included as canonical headers, and/or signed:

  • X-Forwarded-For
  • X-Forwarded-Proto
  • X-Real-IP
  • Connection (at least not included explicitly in the request)

Additionally, it makes no sense to include Authorization as a canonical header as it will cause a recursion issue (the Authorization header includes a signature which, in part, signs the canonical headers themselves).

Signed Headers

The signed headers string is a semi-colon joined list of the header names only of the same sorted, normalized (lowercase) headers in canonical headers. This list must match the canonical headers.

Per the above requirements for canonical headers, the bare minimum signed headers would be:

  • If there is a body, host;content-type;x-amz-sha256
  • If there is no body, host;x-amz-sha256

Signing Key

Unlike AWS2, instead of using the S3 Secret Key directly to sign, a special signing key is generated ("derived") using the S3 Secret Key via the following derivation:

  • First, a key (k⁰) is created by prepending the S3 Secret Key with the string AWS4 (AWS4 + <secret key>)
  • k¹ is then created by HMAC-SHA256 signing the date (in the format YYYYMMDD -- e.g. March 8, 2023 would be 20230308) with k⁰
    • The date must contain (e.g. be the same day as) the timestamp in the signing message!
  • k² is then created by HMAC-SHA256 signing the zone (e.g. us-east1) with k¹
  • k³ is then created by HMAC-SHA256 signing the service with k²
    • The service is always s3.
  • k⁴ is then created by HMAC-SHA256 signing the string aws4_request with k³

k⁴ is then used as the actual signing key for the signing message.

Test Vectors

Given the following request conditions/parameters, it should yield the following signatures if your code is correct:

Vector 1

Parameters

S3 Access Key: JXSXVSSFWLVFGWCUSLLS
S3 Secret Key: IEFfTeUcJffOgbcmSrAXdFTlNHjndsjcTwzNsELU
Request: GET /
Request Headers (Before Signing):

X-AMZ-Content-SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-AMZ-Date: 20230913T213649Z
Host: us-east1.s3.netfire.com

Result

Signature: 2f5f6ec8ca17ccec1ccdb623386319ec648f0157e76cda2bf8bc4104baca076c
Full Request Headers:

X-AMZ-Content-SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-AMZ-Date: 20230913T213649Z
Host: us-east1.s3.netfire.com
Authorization: AWS4-HMAC-SHA256 Credential=88D7KRTO4HXGERCSE4TV/20230913/us-east1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=2f5f6ec8ca17ccec1ccdb623386319ec648f0157e76cda2bf8bc4104baca076c

Vector 2

Parameters

S3 Access Key: NNTIMGQCOARLVMLPBNJM
S3 Secret Key: ZMNNmWZaFbEiFHnOpzRpmAvrpuJggQNskMIDRInq
Request: DELETE /
Request Headers (Before Signing):

X-AMZ-Content-SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-AMZ-Date: 20230913T215826Z
Host: us-east1.s3.netfire.com

Result

Signature: a0695dab908089a0bc3b1e5fcbab8d8b23300b7e5dae905c31f0e94f77b18b4d
Full Request Headers:

X-AMZ-Content-SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-AMZ-Date: 20230913T215826Z
Host: us-east1.s3.netfire.com
Authorization: AWS4-HMAC-SHA256 Credential=NNTIMGQCOARLVMLPBNJM/20230913/us-east1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=a0695dab908089a0bc3b1e5fcbab8d8b23300b7e5dae905c31f0e94f77b18b4d

Python Example

The below will generate the AWS4 signature (and can even be used as a rudimentary S3 client).

#!/usr/bin/env python3

# https://docs.netfire.com/reference/cloud-storage-s3-auth

import argparse
import hashlib
import hmac
import datetime
import os
import re
import urllib.parse
import pprint
##
import requests

DEBUG = False
EP = 'us-east1.s3.netfire.com'
AK = os.environ.get('S3_AK')
SK = os.environ.get('S3_SK')
EPURL = 'https://{0}/'.format(EP)
DATEFMT = '%Y%m%d'
TSFMT = '{0}T%H%M%SZ'.format(DATEFMT)
ZONE = 'us-east1'
SIGNHDR_NAMES = [
    'X-AMZ-Date',
    'Host',
    'X-AMZ-Content-SHA256',
]
AMZ_CKSUM = 'X-AMZ-Content-SHA256'
AMZ_TS = 'X-AMZ-Date'
NO_HDR_NAMES = [
    'x-forwarded-for',
    'x-forwarded-proto',
    'x-real-ip',
    'connection',
    'authorization',
]

EMPTY_SHA = 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'


def get_scope(t: datetime.datetime, zone: str = None):
    if zone is None:
        zone = ZONE
    scope = '/'.join(
        (
            t.strftime(DATEFMT),
            zone,
            's3',
            'aws4_request',
        ),
    )
    if DEBUG:
        print('SCOPE: {0}'.format(scope))
    return (scope)


def get_uri_path(req: requests.Request):
    uri_raw = urllib.parse.urlparse(req.url).path
    if DEBUG:
        print('UNENCODED URL PATH: {0}'.format(uri_raw))
    if uri_raw is None or uri_raw == '':
        return ('')
    encoded = urllib.parse.quote(uri_raw.encode('utf-8'))
    if DEBUG:
        print('ENCODED URL PATH: {0}'.format(encoded))
    return (encoded)


def get_query(req: requests.Request):
    params = []
    if req.params is None:
        return ('')
    if DEBUG:
        print('RAW QUERY PARAMS: {0}'.format(req.params))
    for i in req.params:
        i = i.split('=', 1)
        k = i[0]
        if len(i) > 1:
            v = i[1]
        else:
            v = ''
        params.append('{0}={1}'.format(urllib.parse.quote(k), urllib.parse.quote(v)))
    params = '&'.join(params)
    req.params = params
    if DEBUG:
        print('CANONICAL QUERY PARAMS: {0}'.format(params))
    return (params)


def get_chdrs_shdrs(req: requests.Request):
    chdrs = []
    shdrs = []
    kv = {}
    if isinstance(req.headers, tuple):
        for _, i in enumerate(req.headers):
            k = i[0]
            if len(i) > 1:
                v = i[1]
            else:
                v = ''
            kv[k] = v
    elif isinstance(req.headers, dict):
        kv = dict(req.headers)
    else:
        raise RuntimeError('Unsupported headers type; must be iterable of tuples or dict')
    for k, v in kv.items():
        k = str(k).lower()
        if k in NO_HDR_NAMES:
            continue
        v = re.sub(r'\s+', ' ', str(v))
        shdrs.append(k)
        chdrs.append('{0}:{1}'.format(k, v))
    chdrs.sort()
    shdrs.sort()
    chdrs = '\n'.join(chdrs) + '\n'
    shdrs = ';'.join(shdrs)
    return (chdrs, shdrs)


def get_body_cksum(req: requests.Request):
    if req.data is None or req.data == b'' or req.data == '':
        return (EMPTY_SHA)
    h = hashlib.sha256()
    if isinstance(req.data, bytes):
        h.update(req.data)
    elif isinstance(req.data, str):
        h.update(req.data.encode('utf-8'))
    elif isinstance(req.data, list):
        b = b''
        for i in req.data:
            b += i
        h.update(b)
    else:
        raise RuntimeError('req.data must be bytes or str')
    return (h.hexdigest().lower())


def get_c_req(req: requests.Request):
    h = hashlib.sha256()
    chdrs, shdrs = get_chdrs_shdrs(req)
    c_req = '\n'.join(
        (
            str(req.method).upper(),
            get_uri_path(req),
            get_query(req),
            chdrs,
            shdrs,
            get_body_cksum(req),
        ),
    )
    h.update(c_req.encode('utf-8'))
    return (h.hexdigest().lower())


def get_sigmsg(req: requests.Request, t: datetime.datetime, zone: str = None):
    sigmsg = '\n'.join(
        (
            'AWS4-HMAC-SHA256',
            t.strftime(TSFMT),
            get_scope(t, zone),
            get_c_req(req),
        ),
    )
    return (sigmsg)


def get_sigkey(t: datetime.datetime, zone: str = None):
    if zone is None:
        zone = ZONE
    k0 = 'AWS4' + SK
    k1 = hmac.new(k0.encode('utf-8'), msg=t.strftime(DATEFMT).encode('utf-8'), digestmod='sha256').digest()
    k2 = hmac.new(k1, msg=zone.encode('utf-8'), digestmod='sha256').digest()
    k3 = hmac.new(k2, msg=b's3', digestmod='sha256').digest()
    k4 = hmac.new(k3, msg=b'aws4_request', digestmod='sha256').digest()
    return (k4)


def get_sig(req: requests.Request, t: datetime.datetime, zone: str = None):
    sigmsg = get_sigmsg(req, t, zone)
    sigkey = get_sigkey(t, zone)
    sig = hmac.new(sigkey, msg=sigmsg.encode('utf-8'), digestmod='sha256').hexdigest().lower()
    return (sig)


def get_auth(req: requests.Request, t: datetime.datetime, zone: str = None):
    _, shdrs = get_chdrs_shdrs(req)
    h = ' '.join(
        (
            'AWS4-HMAC-SHA256',
            ','.join(
                (
                    'Credential={0}/{1}'.format(AK, get_scope(t, zone)),
                    'SignedHeaders={0}'.format(shdrs),
                    'Signature={0}'.format(get_sig(req, t, zone)),
                )
            )
        )
    )
    hdr = {'Authorization': h}
    return (hdr)


def parseArgs():
    args = argparse.ArgumentParser(description = 'Hack at S3 requests')
    args.add_argument(
        '-d', '--debug',
        action='store_true',
        dest='debug',
        help='If specified, enable debug mode. WILL spit out a lot of info, including your secret key.',
    )
    args.add_argument(
        '-e', '--endpoint',
        dest='ep',
        default='us-east1.s3.netfire.com',
        help='The endpoint (DNS only). Default: %(default)s',
    )
    args.add_argument(
        '-p', '--path',
        dest='path',
        default='/',
        help='The URL path. Default: %(default)s',
    )
    args.add_argument(
        '-m', '--method',
        dest='method',
        default='GET',
        choices=['GET', 'PUT', 'DELETE', 'POST', 'PATCH'],
        help='The request method. Default: %(default)s',
    )
    args.add_argument(
        '-P', '--param',
        dest='params',
        action='append',
        help='arg=value pairs for URL query parameters; specify multiple times.',
    )
    args.add_argument(
        '-H', '--header',
        dest='hdrs',
        action='append',
        help='arg=value pairs for headers; specify multiple times.',
    )
    args.add_argument(
        '-b', '--body',
        dest='infile',
        help='Path to a file to use for the HTTP request body.',
    )
    args.add_argument(
        '-t', '--time',
        dest='time',
        default=datetime.datetime.utcnow().strftime(TSFMT),
        help='The timestamp to use for the request. Default: %(default)s',
    )
    args.add_argument(
        '-z', '--zone',
        dest='zone',
        default='us-east1',
        help='The S3 zone. Default: %(default)s',
    )
    return(args)


def main():
    global EP
    global EPURL
    global ZONE
    global DEBUG
    args=parseArgs().parse_args()
    DEBUG = args.debug
    t = datetime.datetime.strptime(args.time, TSFMT)
    body = None
    if args.infile is not None:
        with open(args.infile, 'rb') as fh:
            body = fh.read()
    EP = args.ep
    zone = args.zone
    ZONE = args.zone
    EPURL = 'https://{0}{1}'.format(args.ep, args.path)
    hdrs = {}
    params = args.params
    if args.hdrs is not None:
        for i in args.hdrs:
            v = i.split('=', 1)
            hdrs[v[0]] = v[1]
    # if args.params is not None:
    #     params = '&'.join(args.params)

    if args.debug:
        print('ARGS:')
        pprint.pprint(vars(args))
        print('AK: {0}'.format(AK))
        print('SK: {0}'.format(SK))
        print('PARAMS: {0}'.format(params))
    ###############################
    if body is not None:
        h = hashlib.sha256()
        if isinstance(body, bytes):
            h.update(body)
        elif isinstance(body, str):
            h.update(body.encode('utf-8'))
        else:
            raise RuntimeError('body must be bytes or str')
        hdrs[AMZ_CKSUM] = h.hexdigest().lower()
    else:
        hdrs[AMZ_CKSUM] = EMPTY_SHA
    hdrs[AMZ_TS] = t.strftime(TSFMT)
    if args.debug:
        print('BODY CHECKSUM: {0}'.format(hdrs[AMZ_CKSUM]))
        print('TIMESTAMP HEADER: {0}'.format(str(t)))
    hdrs['Host'] = EP
    req = requests.Request(
        method=args.method,
        url=EPURL,
        headers=hdrs,
        data=body,
        params=params,
    )
    auth_hdr = get_auth(req, t, zone=zone)
    req.headers.update(auth_hdr)
    prepped = req.prepare()
    if DEBUG:
        print('FINAL URL: {0}'.format(prepped.url))
    s = requests.Session()
    resp = s.send(prepped)

    print()
    print('STATUS CODE: {0}'.format(resp.status_code))
    print()
    print('REQUEST HEADERS:')
    for k, v in resp.request.headers.items():
        print('{0}: {1}'.format(k, v))
    print()
    print('RESPONSE HEADERS:')
    for k, v in resp.headers.items():
        print('{0}: {1}'.format(k, v))
    print()
    print('RESPONSE BODY:')
    print(resp.content.decode('utf-8'))
    return (None)


if __name__ == '__main__':
    main()