What Is S3 Bucket and How to Access It (Part 1)

By: Noga Yam Amitai

May 12, 2021

S3 bucket misconfigurations account for 16% of all cloud security breaches. While some of this can be chalked up to inexperience or human error, that’s not the only problem going on behind the scenes.

This two-part series will look in-depth at what is an s3 bucket , how AWS handles access rights and permissions, and a new Panoptica Python tool that will provide some visibility and control over securing your public S3 buckets.

What is an S3 Bucket? 

S3 stands for simple storage service, and it is AWS’s cloud storage service. S3 provides the ability to store, retrieve, access, and back up any amount of data at any time and place.

As S3 is object-based storage, this means that all data is stored as objects.

Each object has three main components; the object’s content, the object’s unique identifier, and the object’s metadata (including its name, size, URL).

An Object cannot be independent, it must exist within a bucket. There can be hundreds of buckets in each Amazon account and within each bucket, there can be hundreds of objects. 

How to Access Objects Within a AWS Bucket

Access to a bucket is granted in the same way as with any other AWS resources – you need an explicit allow and no denies in order to be given access.

The explicit allow can be given in three ways – bucket policy, bucket ACL, and object ACL.

S3 Bucket policy: This is a resource-based AWS Identity and Access Management (IAM) policy. You add a bucket policy to a bucket to grant other AWS accounts or IAM users access permissions to the bucket and the objects inside it. Object permissions apply only to the objects that the bucket owner creates.

S3 Bucket ACL/Object ACL: This is a sub-resource that is attached to every S3 bucket and object. It defines which AWS accounts or groups are granted access. If it defines access as public, that will allow everyone permissions, whereas if it defines access only to an authenticated user group, this means anyone with an AWS account will have permissions. It also defines the type of access these users have, such as read or write access.

When you create a bucket or an object, Amazon S3 creates a default ACL that grants the resource owner full control over the resource.

Bucket ACL

Object ACL

Block Public Access

Creating Your Own S3 Bucket Permissions

Amazon provides the ability to define settings for blocking access points, buckets, and accounts to help you to manage public access to Amazon S3 resources. By default, public access to new buckets, access points, and objects is not allowed.

The Block public access settings consist of four options that you can apply in any combination to a bucket or to an entire AWS account. If you apply them to a whole AWS account, these settings will apply to every bucket in the account.

Here are the four options, in the same order as on the console, if you’re looking to manage and access S3 bucket using AWS cli names:

BlockPublicAcls – while set to TRUE, no new ACL definitions are allowed, but existing ones still apply. Meaning, if there is a bucket with an ACL granting public access, the BlockPublicAcls is not going to affect it.

IgnorePublicAcls - while set to TRUE, it causes Amazon S3 to ignore all public ACLs on a bucket and any objects that it contains. Meaning, all public access granted by a bucket or object ACL does not apply.

BlockPublicPolicy - while set to TRUE, no new policies can be attached to the bucket, but existing ones still apply. Meaning, if there is a bucket policy that grants public access, the BlockPublicPolicy is not going to affect it.

RestrictPublicBuckets - while set to TRUE, if there is a policy that allows public access, the access is restricted only to AWS service principals and authorized users within the bucket owner's account.

AWS Bucket Access

AWS evaluates the access scope for each bucket and presents this in the management console.

There are four access options – Public, Objects can be public, Only authorized users of this account, and Bucket and objects not public. These are the only potential outcomes of any access settings applied.

We have created the following diagram to better explain how AWS evaluates the access:

The problem with AWS’s evaluation

As you can see, these four access options don’t necessarily allow you to provide definitive answers to whether your objects are public or not, and which buckets are secure. While ‘Public’ is a black and white outcome, and so is ‘Bucket and objects not public’, the other two are open to confusion. In particular, the outcome of ‘Objects can be public’ leaves your security teams none the wiser about whether items are accessible or not.

Let’s Break it Down with an S3 Bucket Example

Let us assume we have a developer who works with a bucket, and in it, they put a folder with objects, using its ACL to make it publicly accessible. At some point, they want to store some sensitive information in the same folder, so they need to make it non-public.

To do so, they use the block public access settings, specifically IgnorePublicACL, and the bucket and everything in it, is no longer public. The official status is ‘Bucket and objects not public’.

Time goes by and our developer leaves the company, and a new developer joins, and takes over their role.

The new developer wants to create a new public folder in the same bucket for the company’s website, so he changes the IgnorePublicACL back to false, and grants public access to the new folder using its ACL. The bucket status has now changed, and is now ‘Objects can be public.’ This new developer is not aware that their actions have changed the status of other information inside the bucket, as they do not get any alert about the old developer’s folder with the sensitive information that has now become public again.