standardization around compression params

[rchincha]

For a given source tar, compression params matter even if the same compression algorithm is used. Often tar.gz layers are produced but different tools set buffer size, compression levels etc differently, so the final layer ends up being different although the source tar is the same and consequently the sha256sum.

As a result, reproducibility and deduplication suffers. Either clients use identical tooling end-to-end (which is unrealistic) or the standards evolve to encode this variability in the spec.

[rchincha]

https://datatracker.ietf.org/doc/html/rfc7231#section-3.1.1.1

"application/vnd.oci.image.layer.v1.tar+gzip"

-->

"application/vnd.oci.image.layer.v1.tar+gzip; param1=x; param2=y" etc?

[jonjohnsonjr]

There is not one standard algorithm for encoding DEFLATE. Even with the same parameters, different gzip implementations will produce different archives. If you want to record exactly how a layer was produced, the place for that would be in something like an SBOM, not in the mediaType.

[sudo-bmitch]

This might be another case for pushing uncompressed content in the descriptor, and compressing at the transport level.

[rchincha]

There is not one standard algorithm for encoding DEFLATE

Yup.

Also requiring another optional thing (SBOM) for reproducibility?

[rchincha]

Closing this due to the following:

1. OCI image-spec is a spec and a standard but there are limits

2. The spec itself is a lot of JSON and there is no canonical JSON to begin with

3. Output of compression is affected by the algorithm and parameters so you really have to pick and stick with a certain tool that produces the OCI layers (tar and tar.gz). Also make sure you record the tool and version used in order to reproduce the bits.