s3 tests with ACLs on objects failing due to amazon policy change

Description

the 5.3 build is blowing up on github actions, wonder if it's related to this?
https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/

5.4 and 6.0 aren't but they use the s3 ext v2.0

https://github.com/lucee/Lucee/actions/runs/4934461737

Environment

None

Activity

Show:

Brad Wood 15 May 2023 at 01:09

Copying and pasting Slack Thread here:


AWS did make a major change to how S3 buckets work about 2 weeks ago. Any new buckets you create will now always automatically

  • be set to private

  • not allow ACLs to be set

This is a breaking change and if the Lucee tests are creating any sort of temp buckets and then setting items into them, additional work is now required to get a bucket that is

  • public by default

  • or allows ACLs

I had to update the Ortus s3sdk library last week as its tests started failing due to this (edited) 

Glancing at the error messages you put in the ticket, that's exactly what you're hitting slightly smiling face

I know what needs to change in S3, but I'm not sure how Lucee interacts with S3.  The Ortus S3sdk just makes directly HTTP calls to the S3 REST API.  If Lucee is bundling the Java sdk for S3, I'm not 100% sure what the changes will look like for you.

You will need to change the objectOwnership of the bucket from its new default of BucketOwnerEnforced (which means the owner of the bucket enforces the ACLs, and objects in the bucket are not allowed to set their own ACL to ObjectWriter .  This can be done when the bucket is created. (It's the x-amz-object-ownership HTTP header in the REST API)

Secondly, if you intend to set any public ACLs, you'll need to modify one or all of these 4 bucket settings

  • BlockPublicAcls

  • IgnorePublicAcls

  • BlockPublicPolicy

  • RestrictPublicBuckets

The annoying this is these can NOT be set when creating the bucket (at least not in the AWS S3 REST API), so in my S3sdk I had to made a second call to set the PublicAccessBlockConfiguration settings on the bucket.  Once this is done, THEN the bucket can also be updated to have a public default ACL as well if you need (this is 3 total calls if you're keeping track) (edited) 

If you're using the Java S3 SDK, they may have this all wrapped up into a single call to the SDK.  That's how I did it in the Ortus SDK to keep backwards compat, but basically that's what you need to do in order to be able to set object ACLs in a new bucket.

Zac Spitzer 13 May 2023 at 16:59

for 5.3 with the older v.0.9.4 s3 ext there’s a lot more errors

https://github.com/lucee/Lucee/actions/runs/4967915274

6.0 and 5.4 using the v2 / awslib s3 ext now pass

Zac Spitzer 13 May 2023 at 16:29

Zac Spitzer 13 May 2023 at 15:56

The bucket does not allow ACLs;error-code:AccessControlListNotSupported

Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting;error-code:InvalidBucketAclWithObjectOwnership;could not create the bucket [bbfc9c151bf], please consult the following website to learn about Bucket Restrictions and limitations: https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html;Error:vu8pX/O8NogAXVsx1QacPAbg0EVZ3MmfJI32c+W8TCdriwel8p2x8OtHDEfZ7cg47y4+4YPDiBc=

Details

Assignee

Reporter

Priority

Labels

New Issue warning screen

Before you create a new Issue, please post to the mailing list first https://dev.lucee.org

Once the issue has been verified, one of the Lucee team will ask you to file an issue

Affects versions

Created 10 May 2023 at 11:31
Updated 1 July 2023 at 15:37

Flag notifications