AWS: Accessing S3 buckets from the internet and from ec2 instances

After reading about endpoints for AWS S3 i was wondering how i can use this feature.

First step was to create a bucket and just tried to access this bucket over the internet:

$ aws s3api create-bucket --bucket my.webtest --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1
{
    "Location": "http://my.webtest.s3.amazonaws.com/"
}

And the i put this location into my browser:

This is like expected, because i did not allow public access for this bucket:

Ok. Let's try this from an EC2 instance:

$ wget http://my.webtest.s3.amazonaws.com/
--2018-12-04 20:09:47--  http://my.webtest.s3.amazonaws.com/
Auflösen des Hostnamen »my.webtest.s3.amazonaws.com (my.webtest.s3.amazonaws.com)«... 52.216.107.108
Verbindungsaufbau zu my.webtest.s3.amazonaws.com (my.webtest.s3.amazonaws.com)|52.216.107.108|:80... verbunden.
HTTP-Anforderung gesendet, warte auf Antwort... 307 Temporary Redirect
Platz: http://my.webtest.s3-eu-west-1.amazonaws.com/[folge]
--2018-12-04 20:09:47--  http://my.webtest.s3-eu-west-1.amazonaws.com/
Auflösen des Hostnamen »my.webtest.s3-eu-west-1.amazonaws.com (my.webtest.s3-eu-west-1.amazonaws.com)«... 52.218.96.155
Verbindungsaufbau zu my.webtest.s3-eu-west-1.amazonaws.com (my.webtest.s3-eu-west-1.amazonaws.com)|52.218.96.155|:80... verbunden.
HTTP-Anforderung gesendet, warte auf Antwort... 403 Forbidden
2018-12-04 20:09:47 FEHLER 403: Forbidden.

This was not like expected, but how should my bucket know, that this access was from an EC2 instance beloging to the same AWS account.

Let's try to access the bucket with aws cli:

$ aws s3 ls
Unable to locate credentials. You can configure credentials by running "aws configure".

To get this working you have to add an IAM role to your EC2 instance. So let's create a new role:

 choose ec2:

and AmazonS3FullAccess:

Move on (without configuring tags)

 And then attach this role to your EC2 instance:

 and choose your new "AccessToS3Role":

After that the aws cli works like expected:

[ec2-user@ip-172-31-2-99 ~]$ aws s3 ls
2018-12-04 20:02:11 my.webtest
[ec2-user@ip-172-31-2-99 ~]$ aws s3 ls my.webtest 
2018-12-04 20:23:12        130 website.json

But still no access via wget possible. This is because the aws cli uses the Amazon API to access the keys which come with the IAM role attached to the ec2 instance. The wget does not know anything about these keys.

Edit: Finally i got the wget (or better: access without using aws cli) working: https://dietrichschroff.blogspot.com/2019/02/aws-accessing-s3-buckets-from-internet.html