---
layout: global
title: S3 API
---


Alluxio supports a [RESTful API](https://docs.alluxio.io/os/restdoc/{{site.ALLUXIO_MAJOR_VERSION}}/proxy/index.html)
that is compatible with the basic operations of the Amazon [S3 API](http://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html){:target="_blank"}.

The Alluxio S3 API should be used by applications designed to communicate with an S3-like storage
and would benefit from the other features provided by Alluxio, such as data caching, data
sharing with file system based applications, and storage system abstraction (e.g., using Ceph
instead of S3 as the backing store). For example, a simple application that downloads reports
generated by analytic tasks can use the S3 API instead of the more complex file system API.

## Limitations and Disclaimers

### Alluxio Filesystem Limitations

Only top-level Alluxio directories are treated as buckets by the S3 API.
- Hence the root directory of the Alluxio filesystem is not treated as an S3 bucket.
    Any root-level objects (eg: `alluxio://file`) will be inaccessible through the Alluxio S3 API.
- To treat sub-directories as a bucket, the separator `:` must be used in the bucket name (eg: `s3://sub:directory:bucket/file`).
  - **Note that this is purely a convenience feature and hence is not returned by API Actions such as ListBuckets.**

Alluxio uses `/` as a reserved separator. Therefore, any S3 paths with objects or folders named `/`
(eg: `s3://example-bucket//`) will cause undefined behavior. For additional limitations on object key names please check this page: [Alluxio limitations]({{ '/en/reference/Troubleshooting.html' | relativize_url }}#file-path-limitations)

### No Bucket Virtual Hosting

[Virtual hosting of buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html){:target="_blank"} is not supported
in the Alluxio S3 API. Therefore, S3 clients must utilize [path-style requests](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access){:target="_blank"}
(i.e: `http://s3.amazonaws.com/{bucket}/{object}` and NOT `http://{bucket}.s3.amazonaws.com/{object}`).

### S3 Writes Implicitly Overwrite

As described in the AWS S3 docs for [PutObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html){:target="_blank"}:
> _Amazon S3 is a distributed system. If it receives multiple write requests for the same object simultaneously, it overwrites all but the last object written._
> _Amazon S3 does not provide object locking; if you need this, make sure to build it into your application layer or use versioning instead._
- Note that at the moment the Alluxio S3 API does not support object versioning

Alluxio S3 will overwrite the existing key and the temporary directory for multipart upload.

### Folders in ListObjects(V2)

All sub-directories in Alluxio will be returned in ListObjects(V2) as 0-byte folders. This behavior is in accordance
with if you used the AWS S3 console to create all parent folders for each object.

### Tagging & Metadata Limits

User-defined tags on buckets & objects are limited to 10 and obey the [S3 tag restrictions](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-tagging.html){:target="_blank"}.
- Set the property key `alluxio.proxy.s3.tagging.restrictions.enabled=false` to disable this behavior.

The maximum size for user-defined metadata in PUT-requests is 2KB by default in accordance with [S3 object metadata restrictions](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html){:target="_blank"}.
- Set the property key `alluxio.proxy.s3.header.metadata.max.size` to change this behavior.

### Performance Implications

The S3 API leverages the [Alluxio REST proxy]({{ '/en/api/Java-API.html#rest-api' | relativize_url }})
, introducing an additional network hop for Alluxio clients. For optimal performance,
it is recommended to run the proxy server and an Alluxio worker on each compute node.
It is also recommended to put all the proxy servers behind a load balancer.

## Global request headers
<table class="table table-striped">
  <tr>
    <th>Header</th>
    <th>Content</th>
    <th>Description</th>
  </tr>
  <tr>
    <td><a href="https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html" target="_blank">Authorization</a></td>
    <td>AWS4-HMAC-SHA256
    Credential=<b>{user}</b>/...,
    SignedHeaders=...,
    Signature=...</td>
    <td>There is currently no support for access & secret keys in the Alluxio S3 API.
    The only supported authentication scheme is the SIMPLE
    authentication type. By default, the user that is used to perform any operations is the user that was used to
    launch the Alluxio proxy process.
    <br/><br/>
    Therefore this header is used exclusively to specify an Alluxio ACL username to perform an
    operation with. In order to remain compatible with other S3 clients, the header is still
    expected to follow the
    <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html" target="_blank">AWS Signature Version 4</a>
    format.
    <br/><br/>
    When supplying an access key to an S3 client, put the intended Alluxio ACL username.
    The secret key is unused so you may use any dummy value.</td>
  </tr>
</table>

## Supported S3 API Actions

The following table describes the support status for current [S3 API Actions](https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations.html){:target="_blank"}:

<table class="table table-striped">
  <tr>
    <th>S3 API Action</th>
    <th>Supported Headers</th>
    <th>Supported Query Parameters</th>
  </tr>
{% for item in site.data.table.s3-api-supported-actions %}
  <tr>
    <td><a href="https://docs.aws.amazon.com/AmazonS3/latest/API/API_{{ item.action }}.html" target="_blank">{{ item.action }}</a></td>
    <td>
      {% assign headers = item.headers | split: "|" %}
      {% if headers.size == 0 %}
      N/A
      {% else %}
      <ul style="list-style-type:none;margin:0;padding:0">
      {% for header in headers %}
      {% if forloop.last %}
      <li>{{ header }}</li>
      {% else %}
      <li>{{ header }},</li>
      {% endif %}
      {% endfor %}
      </ul>
      {% endif %}
    </td>
    <td>
      {% assign params = item.queryParams | split: "|" %}
      {% if params.size == 0 %}
      N/A
      {% else %}
      <ul style="list-style-type:none;margin:0;padding:0">
      {% for param in params %}
      {% if forloop.last %}
      <li>{{ param }}</li>
      {% else %}
      <li>{{ param }},</li>
      {% endif %}
      {% endfor %}
      </ul>
      {% endif %}
    </td>
  </tr>
{% endfor %}
</table>

## Property Keys

The following table contains the configurable
[Alluxio property keys]({{ '/en/reference/Properties-List.html' | relativize_url }})
which pertain to the Alluxio S3 API.

<table class="table table-striped">
<tr><th>Property Name</th><th>Default</th><th>Description</th></tr>
{% for item in site.data.table.common-configuration %}
  {% if item.propertyName contains "alluxio.proxy.s3" %}
  <tr>
    <td><a class="anchor" name="{{ item.propertyName }}"></a> {{ item.propertyName }}</td>
    <td>{{ item.defaultValue }}</td>
    <td>{{ site.data.table.en.common-configuration[item.propertyName] }}</td>
  </tr>
  {% endif %}
{% endfor %}
</table>

## Example Usage

### S3 API Actions

{% navtabs s3_api_actions %}
{% navtab AWS CLI %}
You can use the [AWS command line interface](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html){:target="_blank"}
to send S3 API requests to the Alluxio S3 API. Note that you will have to provide the `--endpoint` parameter
to specify the location of the Alluxio S3 REST API with the server's base URI included
(i.e: `--endpoint "http://{alluxio.proxy.web.hostname}:{alluxio.proxy.web.port}/api/v1/s3/"`).

As a pre-requisite for operations which involve the `Authorization` header you may need to
[configure AWS credentials](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html){:target="_blank"}.
- See the [Authorization header]({{ '/en/api/S3-API.html#global-request-headers' | relativize_url }})
  for details on how Alluxio uses this header

```shell
$ aws configure --profile alluxio-s3
AWS Access Key ID [None]: {user}
AWS Secret Access Key [None]: {dummy value}
Default region name [None]:
Default output format [None]:
```
{% endnavtab %}
{% navtab REST Clients %}
You can directly use any HTTP client to send S3 API requests to the Alluxio S3 API.
Note that the base URI for the Alluxio S3 API's REST server is `/api/v1/s3/`
(i.e: your requests should be directed to `"http://{alluxio.proxy.web.hostname}:{alluxio.proxy.web.port}/api/v1/s3/"`).

At the moment, access key and secret key validation does not exist for the Alluxio S3 API.
Therefore the [Authorization header]({{ '/en/api/S3-API.html#global-request-headers' | relativize_url }})
is used purely to specify the intended user to perform a request. The header follows the
[AWS Signature Version 4](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html){:target="_blank"} format.

```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." ...
```
{% endnavtab %}
{% endnavtabs %}

#### AbortMultipartUpload

See [AbortMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html){:target="_blank"} on AWS

{% navtabs abort_multipart_upload %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "multipart_copy.txt_6367cf96-ea4e-4447-b931-c5bc91200375/",
            "LastModified": "2022-05-03T13:00:13.429000+00:00",
            "Size": 0
        },
        {
            "Key": "multipart_copy.txt_6367cf96-ea4e-4447-b931-c5bc91200375/1",
            "LastModified": "2022-05-03T13:00:13.584000+00:00",
            "Size": 27040
        },
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:55:01.925000+00:00",
            "Size": 27040
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api abort-multipart-upload \
  --bucket=testbucket --key=multipart_copy.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375

$ % aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:55:01.925000+00:00",
            "Size": 27040
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:45:17 GMT
Content-Type: application/xml
Content-Length: 583
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Marker/>
  <Prefix/>
  <IsTruncated>false</IsTruncated>
  <Name>testbucket</Name>
  <Contents>
    <Key>multipart.txt_6367cf96-ea4e-4447-b931-c5bc91200375/</Key>
    <Size>0</Size>
    <LastModified>2022-05-03T16:44:17.490Z</LastModified>
  </Contents>
  <Contents>
    <Key>multipart.txt_6367cf96-ea4e-4447-b931-c5bc91200375/1</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T16:44:17.715Z</LastModified>
  </Contents>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X DELETE "http://localhost:39999/api/v1/s3/testbucket/multipart.txt?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375"
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 23:45:30 GMT
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:45:36 GMT
Content-Type: application/xml
Content-Length: 318
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Marker/>
  <Prefix/>
  <IsTruncated>false</IsTruncated>
  <Name>testbucket</Name>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>
```
{% endnavtab %}
{% endnavtabs %}

#### CompleteMultipartUpload

See [CompleteMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html){:target="_blank"} on AWS

{% navtabs complete_multipart_upload %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api complete-multipart-upload \
  --bucket=testbucket --key=multipart.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375 \
  --multipart-upload="Parts=[{PartNumber=1},{PartNumber=2}]"
{
    "Location": "/testbucket/multipart.txt",
    "Bucket": "testbucket",
    "Key": "multipart.txt",
    "ETag": "911df44b7ff57801ca8d74568e4ebfbe"
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api head-object \
  --bucket=testbucket --key=multipart.txt
{
    "LastModified": "2022-05-03T20:01:43+00:00",
    "ContentLength": 27040,
    "ETag": "0cc175b9c0f1b6a831c399e269772661",
    "ContentType": "application/octet-stream",
    "Metadata": {}
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ cat complete_upload.xml

<CompleteMultipartUpload xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
   <Part>
      <PartNumber>1</PartNumber>
   </Part>
   <Part>
      <PartNumber>2</PartNumber>
   </Part>
</CompleteMultipartUpload>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -H "Content-Type: application/xml" -d "@complete_upload.xml" \
  -X POST "http://localhost:39999/api/v1/s3/testbucket/multipart.txt?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375"

Date: Tue, 03 May 2022 23:59:17 GMT
Content-Type: text/xml;charset=utf-8
Transfer-Encoding: chunked
Server: Jetty(9.4.43.v20210629)

<CompleteMultipartUploadResult>
  <Location>/testbucket/multipart.txt</Location>
  <Bucket>testbucket</Bucket>
  <Key>multipart.txt</Key>
  <ETag>911df44b7ff57801ca8d74568e4ebfbe</ETag>
  <Code/>
  <Message/>
</CompleteMultipartUploadResult>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  --head "http://localhost:39999/api/v1/s3/testbucket/multipart.txt"
HTTP/1.1 200 OK
Date: Wed, 04 May 2022 00:00:40 GMT
Last-Modified: Tue, 03 May 2022 23:59:18 GMT
ETag: 0cc175b9c0f1b6a831c399e269772661
Content-Type: application/octet-stream
Content-Length: 27040
Server: Jetty(9.4.43.v20210629)
```
{% endnavtab %}
{% endnavtabs %}

#### CopyObject

See [CopyObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html){:target="_blank"} on AWS

{% navtabs copy_object %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api copy-object \
  --copy-source=testbucket/test.txt --bucket=testbucket --key=test_copy.txt
{
    "CopyObjectResult": {
        "ETag": "911df44b7ff57801ca8d74568e4ebfbe",
        "LastModified": "2022-05-03T11:37:16.015000+00:00"
    }
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:35:59.243000+00:00",
            "Size": 27040
        },
        {
            "Key": "test_copy.txt",
            "LastModified": "2022-05-03T11:37:16.185000+00:00",
            "Size": 27040
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -H "x-amz-copy-source: testbucket/test.txt" \
  -X PUT http://localhost:39999/api/v1/s3/testbucket/test_copy.txt
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:50:07 GMT
Content-Type: application/xml
Content-Length: 135
Server: Jetty(9.4.43.v20210629)

<CopyObjectResult>
  <ETag>911df44b7ff57801ca8d74568e4ebfbe</ETag>
  <LastModified>2022-05-03T14:50:07.781Z</LastModified>
</CopyObjectResult>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:50:26 GMT
Content-Type: application/xml
Content-Length: 434
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Contents>
    <Key>test_copy.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:50:07.790Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>
```
{% endnavtab %}
{% endnavtabs %}

#### CreateBucket

See [CreateBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html){:target="_blank"} on AWS

{% navtabs create_bucket %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api create-bucket \
  --bucket=testbucket

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-buckets
{
    "Buckets": [
        {
            "Name": "testbucket",
            "CreationDate": "2022-05-03T11:32:34.156000+00:00"
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X PUT http://localhost:39999/api/v1/s3/testbucket

HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:35:05 GMT
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:35:23 GMT
Content-Type: application/xml
Content-Length: 161
Server: Jetty(9.4.43.v20210629)

<ListAllMyBucketsResult>
  <Buckets>
    <Bucket>
      <Name>testbucket</Name>
      <CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>
```
{% endnavtab %}
{% endnavtabs %}

#### CreateMultipartUpload

See [CreateMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateMultipartUpload.html){:target="_blank"} on AWS

{% navtabs create_multipart_upload %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api create-multipart-upload \
  --bucket=testbucket --key=multipart.txt
{
    "Bucket": "testbucket",
    "Key": "multipart.txt",
    "UploadId": "6367cf96-ea4e-4447-b931-c5bc91200375"
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X POST "http://localhost:39999/api/v1/s3/testbucket/multipart.txt?uploads"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:41:26 GMT
Content-Type: application/xml
Content-Length: 147
Server: Jetty(9.4.43.v20210629)

<InitiateMultipartUploadResult>
  <Bucket>testbucket</Bucket>
  <Key>multipart.txt</Key>
  <UploadId>6367cf96-ea4e-4447-b931-c5bc91200375</UploadId>
</InitiateMultipartUploadResult>
```
{% endnavtab %}
{% endnavtabs %}

#### DeleteBucket

See [DeleteBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucket.html){:target="_blank"} on AWS

{% navtabs delete_bucket %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-buckets
{
    "Buckets": [
        {
            "Name": "tempbucket",
            "CreationDate": "2022-05-03T11:55:58.134000+00:00"
        },
        {
            "Name": "testbucket",
            "CreationDate": "2022-05-03T11:32:34.156000+00:00"
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api delete-bucket \
  --bucket=tempbucket

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-buckets
{
    "Buckets": [
        {
            "Name": "testbucket",
            "CreationDate": "2022-05-03T11:32:34.156000+00:00"
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:43:20 GMT
Content-Type: application/xml
Content-Length: 254
Server: Jetty(9.4.43.v20210629)

<ListAllMyBucketsResult>
  <Buckets>
    <Bucket>
      <Name>tempbucket</Name>
      <CreationDate>2022-05-03T14:43:03.651Z</CreationDate>
    </Bucket>
    <Bucket>
      <Name>testbucket</Name>
      <CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X DELETE http://localhost:39999/api/v1/s3/tempbucket
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 21:43:25 GMT
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:43:28 GMT
Content-Type: application/xml
Content-Length: 161
Server: Jetty(9.4.43.v20210629)

<ListAllMyBucketsResult>
  <Buckets>
    <Bucket>
      <Name>testbucket</Name>
      <CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>
```
{% endnavtab %}
{% endnavtabs %}

#### DeleteBucketTagging

See [DeleteBucketTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketTagging.html){:target="_blank"} on AWS

{% navtabs delete_bucket_tagging %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-bucket-tagging \
  --bucket=testbucket
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api delete-bucket-tagging \
  --bucket=testbucket

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-bucket-tagging \
  --bucket=testbucket
{
    "TagSet": []
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X DELETE "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 23:32:26 GMT
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:32:27 GMT
Content-Type: application/xml
Content-Length: 28
Server: Jetty(9.4.43.v20210629)

<Tagging><TagSet/></Tagging>
```
{% endnavtab %}
{% endnavtabs %}

#### DeleteObject

See [DeleteObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html){:target="_blank"} on AWS

{% navtabs delete_object %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "temp.txt",
            "LastModified": "2022-05-03T11:55:01.925000+00:00",
            "Size": 27040
        },
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:54:19.698000+00:00",
            "Size": 27040
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api delete-object \
  --bucket=testbucket --key=temp.txt

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:55:01.925000+00:00",
            "Size": 27040
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:59:27 GMT
Content-Type: application/xml
Content-Length: 540
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>temp.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:50:07.790Z</LastModified>
  </Contents>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X DELETE http://localhost:39999/api/v1/s3/testbucket/temp.txt
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 22:01:56 GMT
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:01:59 GMT
Content-Type: application/xml
Content-Length: 318
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>
```
{% endnavtab %}
{% endnavtabs %}

#### DeleteObjects

See [DeleteObjects](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html){:target="_blank"} on AWS

{% navtabs delete_objects %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \
  --bucket=tempbucket
{
    "Contents": [
        {
            "Key": "foo.txt",
            "LastModified": "2022-05-03T11:57:00.767000+00:00",
            "Size": 27040
        },
        {
            "Key": "temp.txt",
            "LastModified": "2022-05-03T11:56:11.245000+00:00",
            "Size": 27040
        },
        {
            "Key": "temp2.txt",
            "LastModified": "2022-05-03T11:56:31.414000+00:00",
            "Size": 27040
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api delete-objects \
  --bucket=tempbucket --delete="Objects=[{Key=temp.txt},{Key=temp2.txt}]"

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \
  --bucket=tempbucket
{
    "Contents": [
        {
            "Key": "foo.txt",
            "LastModified": "2022-05-03T11:57:00.767000+00:00",
            "Size": 27040
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/tempbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:59:27 GMT
Content-Type: application/xml
Content-Length: 540
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>foo.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:59:05.906Z</LastModified>
  </Contents>
  <Contents>
    <Key>temp.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:58:58.204Z</LastModified>
  </Contents>
  <Contents>
    <Key>temp2.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:59:01.987Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>tempbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

$ cat delete.xml
<Delete xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Object>
    <Key>temp.txt</Key>
  </Object>
  <Object>
    <Key>temp2.txt</Key>
  </Object>
  <Quiet>false</Quiet>
</Delete>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -H "Content-Type: application/xml" \
  -X POST --data "@delete.xml" "http://localhost:39999/api/v1/s3/testbucket?delete"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:56:17 GMT
Content-Type: application/xml
Content-Length: 208
Server: Jetty(9.4.43.v20210629)

<DeleteResult>
  <Deleted>
    <Key>temp2.txt</Key>
    <DeleteMarker/>
    <DeleteMarkerVersionId/>
    <VersionId/>
  </Deleted>
  <Deleted>
    <Key>temp.txt</Key>
    <DeleteMarker/>
    <DeleteMarkerVersionId/>
    <VersionId/>
  </Deleted>
</DeleteResult>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/tempbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:28:31 GMT
Content-Type: application/xml
Content-Length: 317
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>foo.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:59:05.906Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>tempbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>
```
{% endnavtab %}
{% endnavtabs %}

#### DeleteObjectTagging

See [DeleteObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjectTagging.html){:target="_blank"} on AWS

{% navtabs delete_object_tagging %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object-tagging \
  --bucket=testbucket --key=test.txt
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api delete-object-tagging \
  --bucket=testbucket --key=test.txt

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object-tagging \
  --bucket=testbucket --key=test.txt
{
    "TagSet": []
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X DELETE "http://localhost:39999/api/v1/s3/testbucket/test.txt?tagging"
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 23:37:46 GMT
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket/test.txt?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:37:47 GMT
Content-Type: application/octet-stream
Content-Length: 28
Server: Jetty(9.4.43.v20210629)

<Tagging><TagSet/></Tagging>
```
{% endnavtab %}
{% endnavtabs %}

#### GetBucketTagging

See [GetBucketTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketTagging.html){:target="_blank"} on AWS

{% navtabs get_bucket_tagging %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-bucket-tagging \
  --bucket=testbucket
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>
```
{% endnavtab %}
{% endnavtabs %}

#### GetObject

See [GetObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html){:target="_blank"} on AWS

{% navtabs get_object %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object \
  --bucket=testbucket --key=test.txt /tmp/test.txt
{
    "LastModified": "2022-05-03T18:55:01+00:00",
    "ContentLength": 27040,
    "ETag": "0cc175b9c0f1b6a831c399e269772661",
    "ContentType": "application/octet-stream",
    "Metadata": {}
}

$ stat /tmp/test.txt
  File: /tmp/test.txt
  Size: 27040       Blocks: 56         IO Block: 4096   regular file
  ...
```

{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/testbucket/test.txt
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:59:43 GMT
Last-Modified: Tue, 03 May 2022 21:47:36 GMT
ETag: 0cc175b9c0f1b6a831c399e269772661
Content-Type: application/octet-stream
Content-Length: 27040
Server: Jetty(9.4.43.v20210629)

................. file contents .................
```
{% endnavtab %}
{% endnavtabs %}

#### GetObjectTagging

See [GetObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTagging.html){:target="_blank"} on AWS

{% navtabs get_object_tagging %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object-tagging \
  --bucket=testbucket --key=test.txt
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>
```
{% endnavtab %}
{% endnavtabs %}

#### HeadBucket

See [HeadBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadBucket.html){:target="_blank"} on AWS

{% navtabs head_bucket %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api head-bucket \
  --bucket=testbucket
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  --head http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 15 Nov 2022 04:49:12 GMT
Content-Type: application/xml
Content-Length: 0
Server: Jetty(9.4.43.v20210629)
```
{% endnavtab %}
{% endnavtabs %}

#### HeadObject

See [HeadObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html){:target="_blank"} on AWS

{% navtabs head_object %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api head-object \
  --bucket=testbucket --key=test.txt
{
    "LastModified": "2022-05-03T18:55:01+00:00",
    "ContentLength": 27040,
    "ETag": "0cc175b9c0f1b6a831c399e269772661",
    "ContentType": "application/octet-stream",
    "Metadata": {}
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  --head http://localhost:39999/api/v1/s3/testbucket/test.txt
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:54:22 GMT
Last-Modified: Tue, 03 May 2022 21:47:36 GMT
ETag: 0cc175b9c0f1b6a831c399e269772661
Content-Type: application/octet-stream
Content-Length: 27040
Server: Jetty(9.4.43.v20210629)
```
{% endnavtab %}
{% endnavtabs %}

#### ListBuckets

See [ListBuckets](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html){:target="_blank"} on AWS

{% navtabs list_buckets %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-buckets
{
    "Buckets": [
        {
            "Name": "testbucket",
            "CreationDate": "2022-05-03T11:32:34.156000+00:00"
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:35:23 GMT
Content-Type: application/xml
Content-Length: 161
Server: Jetty(9.4.43.v20210629)

<ListAllMyBucketsResult>
  <Buckets>
    <Bucket>
      <Name>testbucket</Name>
      <CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>
```
{% endnavtab %}
{% endnavtabs %}

#### ListObjects

See [ListObjects](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjects.html){:target="_blank"} on AWS

{% navtabs list_objects %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:35:59.243000+00:00",
            "Size": 27040
        },
        {
            "Key": "test_copy.txt",
            "LastModified": "2022-05-03T11:37:16.185000+00:00",
            "Size": 27040
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:50:26 GMT
Content-Type: application/xml
Content-Length: 434
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Contents>
    <Key>test_copy.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:50:07.790Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>
```
{% endnavtab %}
{% endnavtabs %}

#### ListMultipartUploads

See [ListMultipartUploads](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListMultipartUploads.html){:target="_blank"} on AWS

{% navtabs list_uploads %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint="http://localhost:39999/api/v1/s3" s3api list-multipart-uploads --bucket "testbucket"
{
    "Uploads": [
        {
            "UploadId": "c4cddf71-914a-4cee-b2af-8cfb7def7d04",
            "Key": "object",
            "Initiated": "2022-07-01T11:21:14.738000+00:00"
        },
        {
            "UploadId": "6367cf96-ea4e-4447-b931-c5bc91200375",
            "Key": "object",
            "Initiated": "2022-07-01T11:18:25.290000+00:00"
        },
        {
            "UploadId": "e111c33b-5c18-4ecd-b543-2849cdbbf22b",
            "Key": "object2",
            "Initiated": "2022-07-01T11:21:25.182000+00:00"
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket?uploads"
HTTP/1.1 200 OK
Date: Fri, 01 Jul 2022 18:23:43 GMT
Content-Type: application/xml
Content-Length: 499
Server: Jetty(9.4.46.v20220331)

<ListMultipartUploadsResult>
  <Bucket>testbucket</Bucket>
  <Upload>
    <Key>object</Key>
    <UploadId>c4cddf71-914a-4cee-b2af-8cfb7def7d04</UploadId>
    <Initiated>2022-07-01T11:21:14.738Z</Initiated>
  </Upload>
  <Upload>
    <Key>object</Key>
    <UploadId>6367cf96-ea4e-4447-b931-c5bc91200375</UploadId>
    <Initiated>2022-07-01T11:18:25.290Z</Initiated>
  </Upload>
  <Upload>
    <Key>object2</Key>
    <UploadId>e111c33b-5c18-4ecd-b543-2849cdbbf22b</UploadId>
    <Initiated>2022-07-01T11:21:25.182Z</Initiated>
  </Upload>
</ListMultipartUploadsResult>
```
{% endnavtab %}
{% endnavtabs %}

#### ListObjectsV2

See [ListObjectsV2](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html){:target="_blank"} on AWS

{% navtabs list_objects_v2 %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:35:59.243000+00:00",
            "Size": 27040
        },
        {
            "Key": "test_copy.txt",
            "LastModified": "2022-05-03T11:37:16.185000+00:00",
            "Size": 27040
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket?list-type=2"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:56:20 GMT
Content-Type: application/xml
Content-Length: 438
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>true</version2>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Contents>
    <Key>test_copy.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:50:07.790Z</LastModified>
  </Contents>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <KeyCount>2</KeyCount>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>
```
{% endnavtab %}
{% endnavtabs %}


#### ListParts

See [ListParts](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListParts.html){:target="_blank"} on AWS

{% navtabs list_parts %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-parts \
  --bucket=testbucket --key=multipart.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375
{
    "Parts": [
        {
            "PartNumber": 1,
            "LastModified": "2022-05-03T12:56:27.775000+00:00",
            "ETag": "",
            "Size": 27040
        }
    ],
    "ChecksumAlgorithm": null,
    "Initiator": null,
    "Owner": null,
    "StorageClass": "STANDARD"
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket/multipart.txt?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:49:05 GMT
Content-Type: application/octet-stream
Content-Length: 314
Server: Jetty(9.4.43.v20210629)

<ListPartsResult>
  <Bucket>/testbucket</Bucket>
  <Key>multipart.txt</Key>
  <UploadId>6367cf96-ea4e-4447-b931-c5bc91200375</UploadId>
  <StorageClass>STANDARD</StorageClass>
  <IsTruncated>false</IsTruncated>
  <Part>
    <PartNumber>1</PartNumber>
    <LastModified>2022-05-03T16:48:56.602Z</LastModified>
    <ETag></ETag>
    <Size>27040</Size>
  </Part>
</ListPartsResult>
```
{% endnavtab %}
{% endnavtabs %}

#### PutBucketTagging

See [PutBucketTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketTagging.html){:target="_blank"} on AWS

{% navtabs put_bucket_tagging %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-bucket-tagging \
  --bucket=testbucket
{
    "TagSet": []
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api put-bucket-tagging \
  --bucket=testbucket --tagging='TagSet=[{Key=key1,Value=val1},{Key=key2,Value=val2}]'

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-bucket-tagging \
  --bucket=testbucket
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:30:25 GMT
Content-Type: application/xml
Content-Length: 28
Server: Jetty(9.4.43.v20210629)

<Tagging><TagSet/></Tagging>

$ cat tags.xml
<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -H "Content-Type: application/xml" \
  -X PUT "http://localhost:39999/api/v1/s3/testbucket?tagging" --data-binary "@tags.xml"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:05 GMT
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>
```
{% endnavtab %}
{% endnavtabs %}

#### PutObject

See [PutObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html){:target="_blank"} on AWS

{% navtabs put_object %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api put-object \
  --bucket=testbucket --key=test.txt --body="${ALLUXIO_HOME}/LICENSE"
{
    "ETag": "911df44b7ff57801ca8d74568e4ebfbe"
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:35:59.243000+00:00",
            "Size": 27040
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X PUT http://localhost:39999/api/v1/s3/testbucket/test.txt -T "${ALLUXIO_HOME}/LICENSE"
HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:47:36 GMT
ETag: 911df44b7ff57801ca8d74568e4ebfbe
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:47:44 GMT
Content-Type: application/xml
Content-Length: 318
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>
```
{% endnavtab %}
{% endnavtabs %}

#### PutObjectTagging

See [PutObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectTagging.html){:target="_blank"} on AWS

{% navtabs put_object_tagging %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object-tagging \
  --bucket=testbucket --key=test.txt
{
    "TagSet": []
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api put-object-tagging \
  --bucket=testbucket --key=test.txt --tagging='TagSet=[{Key=key1,Value=val1},{Key=key2,Value=val2}]'

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object-tagging \
  --bucket=testbucket --key=test.txt
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket/test.txt?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:30:25 GMT
Content-Type: application/xml
Content-Length: 28
Server: Jetty(9.4.43.v20210629)

<Tagging><TagSet/></Tagging>

$ cat tags.xml
<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -H "Content-Type: application/xml" \
  -X PUT "http://localhost:39999/api/v1/s3/testbucket/test.txt?tagging" --data-binary "@tags.xml"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:35:28 GMT
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:39999/api/v1/s3/testbucket/test.txt?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:35:58 GMT
Content-Type: application/octet-stream
Content-Length: 126
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>
```
{% endnavtab %}
{% endnavtabs %}

#### UploadPart

See [UploadPart](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html){:target="_blank"} on AWS

{% navtabs upload_part %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api upload-part \
  --bucket=testbucket --key=multipart.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375 --part-number=1 --body="${ALLUXIO_HOME}/LICENSE"
{
    "ETag": "911df44b7ff57801ca8d74568e4ebfbe"
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-parts \
  --bucket=testbucket --key=multipart.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375
{
    "Parts": [
        {
            "PartNumber": 1,
            "LastModified": "2022-05-03T12:56:27.775000+00:00",
            "ETag": "",
            "Size": 27040
        }
    ],
    "ChecksumAlgorithm": null,
    "Initiator": null,
    "Owner": null,
    "StorageClass": "STANDARD"
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X PUT -T "${ALLUXIO_HOME}/LICENSE" "http://localhost:39999/api/v1/s3/testbucket/multipart.txt?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375&partNumber=1"
HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:51:19 GMT
ETag: "911df44b7ff57801ca8d74568e4ebfbe"
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-parts \
  --bucket=testbucket --key=multipart_copy.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375
{
    "Parts": [
        {
            "PartNumber": 1,
            "LastModified": "2022-05-03T13:00:13.584000+00:00",
            "ETag": "",
            "Size": 27040
        }
    ],
    "ChecksumAlgorithm": null,
    "Initiator": null,
    "Owner": null,
    "StorageClass": "STANDARD"
}
```
{% endnavtab %}
{% endnavtabs %}

#### UplaodPartCopy

See [UploadPartCopy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPartCopy.html){:target="_blank"} on AWS

{% navtabs upload_part_copy %}
{% navtab AWS CLI %}
```shell
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api upload-part-copy \
  --bucket=testbucket --key=object --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375 --part-number=1 --copy-source=testbucket/object
{
    "CopyPartResult": {
        "ETag": "0cc175b9c0f1b6a831c399e269772661"
    }
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-parts \
  --bucket=testbucket --key=object --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375
{
    "Parts": [
        {
            "PartNumber": 1,
            "LastModified": "2022-07-01T11:27:48.942000+00:00",
            "ETag": "",
            "Size": 1
        }
    ],
    "ChecksumAlgorithm": null,
    "Initiator": null,
    "Owner": null,
    "StorageClass": "STANDARD"
}
```
{% endnavtab %}
{% navtab REST Clients %}
```shell
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -H "x-amz-copy-source: testbucket/object" \
  -X PUT 'http://localhost:39999/api/v1/s3/testbucket/object?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375&partNumber=1'
HTTP/1.1 200 OK
Date: Fri, 01 Jul 2022 18:31:34 GMT
Content-Type: application/xml
Content-Length: 78
Server: Jetty(9.4.46.v20220331)

<CopyPartResult><ETag>0cc175b9c0f1b6a831c399e269772661</ETag></CopyPartResult>
```
{% endnavtab %}
{% endnavtabs %}

### Python S3 Client

Tested for Python 2.7.

#### Create a connection
Please note you have to install boto package first.

```shell
$ pip install boto
```

```python
import boto
import boto.s3.connection

conn = boto.connect_s3(
    aws_access_key_id = '',
    aws_secret_access_key = '',
    host = 'localhost',
    port = 39999,
    path = '/api/v1/s3',
    is_secure=False,
    calling_format = boto.s3.connection.OrdinaryCallingFormat(),
)
```

#### Authenticating as a user
By default, authenticating with no access_key_id uses the user that was used to launch the proxy
as the user performing the file system actions.

Set the ```aws_access_key_id``` to a different username to perform the actions under a different user.

#### Create a bucket

```python
bucketName = 'bucket-for-testing'
bucket = conn.create_bucket(bucketName)
```

#### List all buckets owned by the user

Authenticating as a user is necessary to have buckets returned by this operation.

```python
conn = boto.connect_s3(
    aws_access_key_id = 'testuser',
    aws_secret_access_key = '',
    host = 'localhost',
    port = 39999,
    path = '/api/v1/s3',
    is_secure=False,
    calling_format = boto.s3.connection.OrdinaryCallingFormat(),
)

conn.get_all_buckets()
```

#### Put a small object

```python
smallObjectKey = 'small.txt'
smallObjectContent = 'Hello World!'

key = bucket.new_key(smallObjectKey)
key.set_contents_from_string(smallObjectContent)
```

#### Get the small object

```python
assert smallObjectContent == key.get_contents_as_string()
```

#### Upload a large object
Create a 8MB file on local file system.

```shell
$ dd if=/dev/zero of=8mb.data bs=1048576 count=8
```

Then use python S3 client to upload this as an object

```python
largeObjectKey = 'large.txt'
largeObjectFile = '8mb.data'

key = bucket.new_key(largeObjectKey)
with open(largeObjectFile, 'rb') as f:
    key.set_contents_from_file(f)
with open(largeObjectFile, 'rb') as f:
    largeObject = f.read()
```

#### Get the large object

```python
assert largeObject == key.get_contents_as_string()
```

#### Delete the objects

```python
bucket.delete_key(smallObjectKey)
bucket.delete_key(largeObjectKey)
```

#### Initiate a multipart upload

```python
mp = bucket.initiate_multipart_upload(largeObjectKey)
```

#### Upload parts

```python
import math, os

from filechunkio import FileChunkIO

# Use a chunk size of 1MB (feel free to change this)
sourceSize = os.stat(largeObjectFile).st_size
chunkSize = 1048576
chunkCount = int(math.ceil(sourceSize / float(chunkSize)))

for i in range(chunkCount):
    offset = chunkSize * i
    bytes = min(chunkSize, sourceSize - offset)
    with FileChunkIO(largeObjectFile, 'r', offset=offset, bytes=bytes) as fp:
        mp.upload_part_from_file(fp, part_num=i + 1)
```

#### Complete the multipart upload

```python
mp.complete_upload()
```

#### Abort the multipart upload

Non-completed uploads can be aborted.

```python
mp.cancel_upload()
```

#### Delete the bucket

```python
bucket.delete_key(largeObjectKey)
conn.delete_bucket(bucketName)
```
