Announcement: S3 default integrity change

[aBurmeseDev]

In AWS SDK for JavaScript v3.729.0, we released changes to the S3 client that adopts new default integrity protections. For more information on default integrity behavior, please refer to the official SDK documentation. In SDK releases from this version on, clients default to enabling an additional checksum on all Put calls and enabling validation on Get calls.

You can disable default integrity protections for S3. We do not recommend this because checksums are important to S3 integrity posture. Integrity protections can be disabled by setting the config flag to WHEN_REQUIRED, or by using the related AWS shared config file settings or environment variables.

Disclaimer: the AWS SDKs and CLI are designed for usage with official AWS services. We may introduce and enable new features by default, such as these new default integrity protections, prior to them being supported or otherwise handled by third-party service implementations. You can disable the new behavior with the WHEN_REQUIRED value for the request_checksum_calculation and response_checksum_validation configuration options covered in Data Integrity Protections for Amazon S3.

[NGPixel]

Since #6810 was locked for no valid reason by @trivikr, let me point out 2 scenarios where this change negatively affects AWS customers:

• Many users rely on multiple cloud providers / on-premise solutions, whether it's for redundancy, failover or just plain local development. While my production environment could be AWS, I could use minio for local development. My local development / test environment would be completely broken right now, even though my target production environment is AWS S3.

• Users / apps that use a proxy to relay requests to AWS (via the endpoint option) could have security measures in place that only allow certain headers for outgoing requests. With this release, these requests are now failing.

As such, this should have been a major release.

So you can hide behind your ridiculous "we only support AWS services" argument all you want, this release was completely botched, doesn't respect semantic versioning (semver) and goes against basic open source library principles. You can pretend you support open source in your marketing and announcements but your actions here shows otherwise.

[trivikr]

Hi @NGPixel, thank your taking time to summarize use cases which can impact AWS customers.

While my production environment could be AWS, I could use minio for local development

MinIO maintainers are aware of this, and have agreed to support it minio/minio#20845 (comment)

Users / apps that use a proxy to relay requests to AWS (via the endpoint option) could have security measures in place that only allow certain headers for outgoing requests.

When any AWS SDK communicates with the AWS service backend, there's a service contract on certain headers to be set. This is defined in open sourced smithy models. This contract can change based on customer requirements in rare cases like enabling end-to-end data integrity for S3 objects. If it does, the input shape provided by customer or the output shape they receive are not impacted.

If you're using a third party proxy for requests between SDK and AWS service, it introduces a security risk as it will have access to your data being sent to AWS. But if you're using it for some reason, the proxy should be aware of the open source AWS service models so that it doesn't mistakenly remove valid headers. Can you report this issue to your proxy provider? In the meantime, you can pin your SDK version so that your application is not impacted with current or future updates.

As I mentioned on the GitHub issue, this discussion is also for announcement to improve discoverability.
Please create a new feature request or a bug report for the specific concern and we can discuss further.