Amazon AWS S3

 

Amazon Simple Storage Service (S3) is a service offered by Amazon Web Services (AWS) that provides storage for any type of object. This allows AWS S3 to provide storage services for a wide variety of environments. The data is managed by AWS S3 using object storage architecture which provides low latency, high availability, and easy scalability. Each object is given a user-assigned, unique key and organized into buckets. These buckets are managed using the AWS console, the AWS SDK, or the REST API.

 

See Amazon S3 Permissions for details on the required permissions.

 

 

Amazon S3 can be used to replace static web-hosting infrastructure with HTTP client-accessible objects, index document support, and error document support.

 

 

Login

Sets authenticated session for subsequent S3 job steps.

 

 

Job Step Properties

 

Connection: Connection is optional for all Job Steps except Login. Select Connection Data from the drop-down menu and enter a User Account and

Region.

 

User Account: Requires an AWS Access Key ID and Secret Access Key.

 

Username (AWS Access Key)/Password (Secret Access Key).

 

Region: Enter the AWS Region. By default, AWS uses us-east-1.

 

List Objects

List all objects or a subset based on a prefix.

 

 

Job Step Properties

 

Connection: If not specified by the Login Job Step, this field is required. Select Connection Data from the drop-down menu and enter a User Account and

Region.

 

Bucket Name: Required to specify the bucket name.

 

Prefix: A prefix can be used to organize the data stored in S3 buckets by using a string of characters before the object key name. Prefixes allow you to store

data in the same way as file system directories.

 

Delete Object

 

Deletes an object specified by a file key.

 

 

Job Step Properties

 

Connection: If not specified by the Login Job Step, this field is required. Select Connection Data from the drop-down menu and enter a User Account and

Region.

 

Bucket Name: Required to specify the bucket name.

 

File Key: The AWS Object Key. This field can also include a prefix. For example,

 

test/mytestfile

 

Download

 

Downloads the specified file.

 

 

Job Step Properties

 

Connection: If not specified by the Login Job Step, this field is required. Select Connection Data from the drop-down menu and enter a User Account and

Region.

 

Bucket Name: Required to specify the bucket name.

 

File Key:The AWS Object Key. This field can also include a prefix. For example,

 

test/mytestfile

 

Target File Path: Specifies the local file path the job is executing on.

 

Specify a File path specification that is present on the local system the job is executing on

 

 

List Buckets

Returns a list of all buckets owned by the authenticated sender of the request.

 

 

Job Step Properties

 

Connection: If not specified by the Login Job Step, this field is required. Select Connection Data from the drop-down menu and enter a User Account and

Region.

 

 

Upload

 

Uploads the specified file.

 

 

Job Step Properties

 

Connection: If not specified by the Login Job Step, this field is required. Select Connection Data from the drop-down menu and enter a User Account and

Region.

 

Bucket Name: Required to specify the bucket name.

 

File Key:

 

Source File Path: Specifies the local file path the job is executing on.

 

Copy

 

Copy an Object from one location to another.

 

 

Connection: If not specified by the Login Job Step, this field is required. Select Connection Data from the drop-down menu and enter a User Account and

Region.

 

Source Bucket: The name of the bucket where the source object is located.

 

Source Key: The key (path and filename) of the source object to copy.

 

Destination Bucket: The name of the object where the source object will be copied.

 

Destination Key: The key (path and filename) where the source object will be copied.

 

Move

 

Move an Object from one location to another.

 

 

Connection: If not specified by the Login Job Step, this field is required. Select Connection Data from the drop-down menu and enter a User Account and

Region.

 

Source Bucket: The name of the bucket where the source object is located.

 

Source Key: The key (path and filename) of the source object to move.

 

Destination Bucket: The name of the object where the source object will be moved.

 

Destination Key: The key (path and filename) where the source object will be moved.

 

Amazon S3 Permissions

These are the required permissions for several AWS S3 operations.

 

List Buckets: Returns a list of all buckets owned by the user. This operation requires s3:ListAllMyBucketspermission.

 

 

List Objects: You must have READ access to the bucket to use this operation and permission to perform the s3:ListBucket action.

 

Note: Amazon recommends using the newer version, ListObjectsV2, however ListObjects will continue to be supported.

 

 

Copy Objects: Creates a copy of an object that is already stored in Amazon S3.

 

Note: You must have READ access to the source object and WRITE access to the destination bucket.

 

General purpose bucket permissions: You must have permissions in an IAM policy based on the source and destination bucket types in a CopyObject operation.

  • If the source object is in a general purpose bucket, you must have s3:GetObject permission to read the source object that is being copied.

  • If the destination bucket is a general purpose bucket, you must have s3:PutObject permission to write the object copy to the destination bucket.

Directory bucket permissions: You must have permissions in a bucket policy or an IAM identity-based policy based on the source and destination bucket types in a CopyObject operation.

 

  • If the source object that you want to copy is in a directory bucket, you must have the s3express:CreateSession permission in the Action element of a policy to read the object. By default, the session is in the ReadWrite mode. If you want to restrict the access, you can explicitly set the s3express:SessionMode condition key to ReadOnly on the copy source bucket.

  • If the copy destination is a directory bucket, you must have the s3express:CreateSession permission in the Action element of a policy to write the object to the destination. The s3express:SessionMode condition key can't be set to ReadOnly on the copy destination bucket.

 

To use HEAD, you must have the s3:GetObjectpermission. You need the relevant read object (or version) permission for this

operation.

 

For more information, see Actions, Resources, and Condition Keys for Amazon S3 in the Amazon S3 User Guide.

 

For more information about the permissions to S3 API operations by S3 resource types, see Required permissions for Amazon S3 API Operations in the Amazon S3 User Guide.

 

If the object you request doesn't exist, the error that Amazon S3 returns depends on whether you also have the s3:ListBucket permission.

  • If you have the s3:ListBucket permission on the bucket, Amazon S3 returns an HTTP status code 404 Not Found error.

  • If you don’t have the s3:ListBucket permission, Amazon S3 returns an HTTP status code 403 Forbidden error.

DeleteObject:s3:DeleteObject Removes an object from a bucket.

 

Required if you want to remove an object without specifying Version ID in the request.

 

MoveObject: Requires permissions from both CopyObject and DeleteObject.

 

Upload: Requires permissions from CopyObject, List Objects, and List buckets along with the following:

  • s3:AbortMultipartUpload

  • s3:ListBucketMultipartUploads

  • s3:ListMultipartUploadParts

Download: requires permissions from CopyObject, List Objects, and List buckets