Contact Support

S3 Bucket Operations

As you may recall, buckets are containers for objects (which are typically files).

As such, there are some operations that can be performed on/with buckets to manage them.

All requests mentioned here require authentication.

📘

Remember, DNS buckets are demonstrated!

Request URL paths will need to be modified if using path buckets.

Bucket URLs

Aside from list operations, all operations operate on the bucket URL. The format of the bucket URL depends on if you are using DNS bucketing or path bucketing.

DNS Bucket URL

The DNS bucket URL is quite simple, but can only be used for authenticated requests.

It comprises of the bucket name and endpoint (i.e. <scheme>://<bucket name>.<endpoint>/, where <scheme> is one of http or https (https strongly recommended) and is your S3 endpoint).

Path Bucket URL

Path bucket URLs have the benefit of being a little more explicit (which allows for unauthenticated requests) but at the cost of not being quite as "clean-looking" as DNS Bucket URLs.

It comprises of the endpoint, the tenant, the username, and the bucket name (i.e. <scheme>://<endpoint>/<tenant name>:<username>/<bucket name> where <scheme> is one of http or https (https strongly recommended) and is your S3 endpoint).

Creating Buckets

A bucket is created by performing a PUT on the Bucket URL. The following creates a bucket called newbucket.

📘

The <CreateBucketConfiguration>...</CreateBucketConfiguration> should not be strictly necessary.

This is only to apply e.g. region preferences, etc. The below specifies to create the bucket in the us-east region.

# BEGIN REQUEST #
PUT / HTTP/1.1
Host: newbucket.us-east1.s3.netfire.com
Content-Length: 103
X-AMZ-Date: 20230913T222823Z
X-AMZ-Content-SHA256: bf48b1739d03760f5c14d1dcc3aec56977ba485c43a4a34b65c72e614ebcc71e
Authorization: AWS4-HMAC-SHA256 Credential=88D7KRTO4HXGERCSE4TV/20230913/us-east/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=f91337335fa8f074a88671922ffb0dd76d8e7f5a1225abf9601ae30ea9922e13

## BEGIN REQUEST BODY ##
<CreateBucketConfiguration><LocationConstraint>us-east</LocationConstraint></CreateBucketConfiguration>
## END REQUEST BODY ##
# END REQUEST #

# BEGIN RESPONSE #
HTTP/1.1 200 OK
X-AMZ-Request-ID: tx00000e477344a0a3ab7df-0065023787-978641-us-east1
Content-Length: 0
Date: Wed, 13 Sep 2023 22:28:23 GMT
Connection: close
# END RESPONSE #

"Prettified" request body:

<?xml version="1.0"?>
<CreateBucketConfiguration>
  <LocationConstraint>us-east</LocationConstraint>
</CreateBucketConfiguration>

Deleting Buckets

Buckets can be deleted by performing a DELETE on the Bucket URL. The following deletes a bucket called newbucket.

# BEGIN REQUEST #
DELETE / HTTP/1.1
Host: newbucket.us-east1.s3.netfire.com
Content-Length: 0
X-AMZ-Date: 20230913T223354Z
X-AMZ-Content-SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
authorization: AWS4-HMAC-SHA256 Credential=88D7KRTO4HXGERCSE4TV/20230913/us-east/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=df52bbe2bc1b383b2d3f3288c54bf8649f04950dfe40f04d97e9c231156e9199
# END REQUEST #

# BEGIN RESPONSE #
HTTP/1.1 204 No Content
x-amz-request-id: tx000004b332025e63c10b7-00650238d2-978641-us-east1
Date: Wed, 13 Sep 2023 22:33:54 GMT
Connection: close
# END RESPONSE #

Listing Buckets

Your account's buckets can be listed by performing a GET on your endpoint.

# BEGIN REQUEST #
GET / HTTP/1.1
Host: us-east1.s3.netfire.com
Content-Length: 0
X-AMZ-Date: 20230913T223737Z
X-AMZ-Content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=88D7KRTO4HXGERCSE4TV/20230913/us-east/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=fc19555446abd8241f2d7c81b585e65281ec446ad30009ca80dec3c51645e901
# END REQUEST #

# BEGIN RESPONSE #
HTTP/1.1 200 OK
Transfer-Encoding: chunked
x-amz-request-id: tx00000fcfab41feaa846e0-00650239b1-978641-us-east1
Content-Type: application/xml
Date: Wed, 13 Sep 2023 22:37:37 GMT
Connection: close

## BEGIN RESPONSE BODY ##
<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>NetFire$example</ID><DisplayName>Example</DisplayName></Owner><Buckets><Bucket><Name>example</Name><CreationDate>2023-03-06T17:13:58.629Z</CreationDate></Bucket><Bucket><Name>example2</Name><CreationDate>2023-03-08T13:33:20.761Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>
## END RESPONSE BODY ##
# END RESPONSE #

"Prettified" response body:

<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Owner>
    <ID>NetFire$example</ID>
    <DisplayName>Example</DisplayName>
  </Owner>
  <Buckets>
    <Bucket>
      <Name>example</Name>
      <CreationDate>2023-03-06T17:13:58.629Z</CreationDate>
    </Bucket>
    <Bucket>
      <Name>example2</Name>
      <CreationDate>2023-03-08T13:33:20.761Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>

Note that information about our user, the /ListAllMyBucketsResult/Owner/ID, is displayed which correlates to our username, and /ListAllMyBucketsResult/Owner/Name, which (normally) would correlate to a human-friendly name (e.g. Alice Smith, Bob Johnson, etc.) but for documentation purposes here is Example.

Bucket ACLs

If you wish to grant access to buckets to other members in your Tenant, this can be done via ACLs.

📘

You can grant/revoke ACLs by username ($), email address, or S3 Access Key of the user ("grantee")

Note that s3cmd in particular has difficulty with uppercase letters in grantees when setting grants, so if you're using that client you may need to use the email address to specify the grantee.

Getting ACL

To view the current ACL of a bucket, a GET /?acl is performed on your Bucket URL.

# BEGIN REQUEST #
GET /?acl HTTP/1.1
Host: example.us-east1.s3.netfire.com
Content-Length: 0
X-AMZ-Date: 20230913T230621Z
X-AMZ-Content-SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=JXSXVSSFWLVFGWCUSLLS/20230913/us-east1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=e2663bc09eeed65889b4214c9a85154f288e4aed0072841a4da5c01bd3f3607b
# END REQUEST #

# BEGIN RESPONSE #
HTTP/1.1 200 OK
X-AMZ-Request-ID: tx0000055b0c740bb113a48-006502406d-978656-us-east1
Content-Type: application/xml
Content-Length: 449
Date: Wed, 13 Sep 2023 23:06:21 GMT
Connection: close

## BEGIN RESPONSE BODY ##
<?xml version="1.0" encoding="UTF-8"?><AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>NetFire$example</ID><DisplayName>Example</DisplayName></Owner><AccessControlList><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>NetFire$example</ID><DisplayName>Example</DisplayName></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>
## END RESPONSE BODY ##
# END RESPONSE #

"Prettified" response body:

<?xml version="1.0" encoding="UTF-8"?>
<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Owner>
    <ID>NetFire$example</ID>
    <DisplayName>Example</DisplayName>
  </Owner>
  <AccessControlList>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>NetFire$example</ID>
        <DisplayName>Example</DisplayName>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
  </AccessControlList>
</AccessControlPolicy>

Setting/Granting ACL

The below grants FULL_CONTROL of the bucket example to the user with the email [email protected]:

# BEGIN REQUEST #
PUT /?acl HTTP/1.1
Host: example.us-east1.s3.netfire.com
Content-Length: 545
Content-Type: application/xml
X-AMZ-Date: 20230913T231345Z
X-AMZ-Content-SHA256: d0aec4b4054704761f1accca7a7e64e51590ea36d9b8db05eaa032cb017e5207
Authorization: AWS4-HMAC-SHA256 Credential=JXSXVSSFWLVFGWCUSLLS/20230913/us-east1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=c5829fb92a5d4fffcd9fc1ca7993558e97bc46261277918bd0afd8adca767767


## BEGIN REQUEST BODY ##
<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>NetFire$example</ID></Owner><AccessControlList><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>NetFire$example</ID></Grantee><Permission>FULL_CONTROL</Permission></Grant><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="AmazonCustomerByEmail"><EmailAddress>[email protected]</EmailAddress></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>
## END REQUEST BODY ##
# END REQUEST #

# BEGIN RESPONSE #
HTTP/1.1 200 OK
X-AMZ-Request-ID: tx00000db06a10034513ccc-0065024229-978641-us-east1
Content-Type: application/xml
Content-Length: 0
Date: Wed, 13 Sep 2023 23:13:45 GMT
Connection: close
# END RESPONSE #

"Prettified" request body:

<?xml version="1.0"?>
<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Owner>
    <ID>NetFire$example</ID>
  </Owner>
  <AccessControlList>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>NetFire$example</ID>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="AmazonCustomerByEmail">
        <EmailAddress>[email protected]</EmailAddress>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
  </AccessControlList>
</AccessControlPolicy>

Removing/Revoking ACL

To remove permissions for a user, it's the same process as granting -- just omit them from the /AccessControlPolicy/AccessControlList/Grant[] list and update the ACL.

In the example below, we remove the grant we created above. Note that it is simply removed from the ACL definition.

# BEGIN REQUEST #
PUT /?acl HTTP/1.1
Host: example.us-east1.s3.netfire.com
Content-Length: 343
Content-Type: application/xml
X-AMZ-Date: 20230913T233914Z
X-AMZ-Content-SHA256: bdbb2c7107bd782463523494cd47c6a5a9c8d4516e862e277493a9a18fcb4a7b
Authorization: AWS4-HMAC-SHA256 Credential=JXSXVSSFWLVFGWCUSLLS/20230913/us-east1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=e1233ffaa8a805ff0439e993f7eca2ece5df3a53f8b3397a24d53bc5bf9c98d2


## BEGIN REQUEST BODY ##
<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>NetFire$example</ID></Owner><AccessControlList><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>NetFire$example</ID></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>
## END REQUEST BODY ##
# END REQUEST #

# BEGIN RESPONSE #
HTTP/1.1 200 OK
X-AMZ-Request-ID: tx0000045a94588c1844101-0065024822-978656-us-east1
Content-Type: application/xml
Content-Length: 0
Date: Wed, 13 Sep 2023 23:39:14 GMT
Connection: close
# END RESPONSE #

"Prettified" request body:

<?xml version="1.0"?>
<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Owner>
    <ID>NetFire$example</ID>
  </Owner>
  <AccessControlList>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>NetFire$example</ID>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
  </AccessControlList>
</AccessControlPolicy>