Announcement: S3 Default Integrity Change

(github.com)

5 points | by snide6 hours ago

3 comments

  • toomuchtodo6 hours ago
    Heads up: This is/was a breaking change for folks using this with S3 compatible (but not S3) target storage systems.

    Related:

    Latest AWS SDK/CLI incompatible with S3 alternatives - https://news.ycombinator.com/item?id=42740910 - Jan 2025

    Config hotfix:

    https://github.com/boto/boto3/issues/4400#issuecomment-26007...

      s3_client = boto3.client(
           's3',
           endpoint_url=os.getenv('S3_ENDPOINT'),
           
      config=Config(request_checksum_calculation="when_required", response_checksum_validation="when_required")
     )
    • jcrben6 hours ago
      The issue is also discussed in the linked issue.

      Maybe this will trigger a rethink in how s3 compatible storage systems work.

      • toomuchtodo6 hours ago
        S3 became an unofficial de facto standard due to their size, but the relationship will become more adversarial as AWS attempts to prevent leakage. Therefore, there are two parts to this:

        1. Storage systems not S3 need to get on the same page for customer benefit wrt protocol and standards for object storage. 2. Users should default to non AWS client side tooling if they want to remain vendor agnostic, otherwise AWS will drag users for their benefit and purposes.

        This is really no different than the OpenTofo [Terraform fork] & OpenBao [Vault fork] situation imho. Instead of "S3 compatible," it's time to fork and make the de factor standard an open standard, leaving AWS users to have to shim if breaking forward changes occur.

        (and before S3, it was mogileFS: https://github.com/mogilefs/mogilefs-docs [which powered LiveJournal object storage almost two decades ago; omg files!])

  • skywhopper5 hours ago
    Lots of folks in the comments who aren’t practicing good code safety when using a client library that’s not from the vendor of their storage service.
  • 6 hours ago
    undefined