Paketo jammy-full builder release v0.3.388 fails with Docker error "max depth exceeded"

[outcomes-michael-pinnegar]

Release v0.3.388 causes our CI build to fail with a "max depth exceeded" error from Docker. Docker has a hard, fixed limit of 127 layers. You can't have more than that. Release v0.3.387 does NOT produce that error.

I believe this issue just started cropping up because builder-jammy-full/0.3.386 and 0.3.387 had 114 layers, latest (0.3.388) has 121 layers. Basically you only have 6 more layers of overhead before hitting the hard limit set by docker. I don't know why or how Paketo seems to be modifying the number of layers in the builder image, but apparently it does, because the failure is sensitive to the layer count in the builder image.

Expected Behavior

Build should proceed normally

Current Behavior

Digest: sha256:858f4b789918283d33a5885d3c8f56be9b3c581e61fa76158b52e1c7d5eb758f
Status: Downloaded newer image for paketobuildpacks/builder-jammy-full:latest
ERROR: failed to build: failed to write image to the following tags: [pack.local/builder/66776672666266706164:latest: loading image "pack.local/builder/66776672666266706164:latest". first error: embedded daemon response: max depth exceeded]

Possible Solution

Reduce the number of layers in the produced full-jammy docker image. A simple way to do this is to reduce the number of commands in your layers, and I believe Multistage builds is a more advanced way to shrink the number of layers, but I haven't used it before.

Steps to Reproduce

Run pack build image_name:1.0.0 --env 'BP_EMBED_CERTS=true' --env 'BP_RUNTIME_CERT_BINDING_DISABLED=true' --publish --post-buildpack paketobuildpacks/elastic-apm:latest

I actually haven't tried to reproduce this outside of our CI environment, but this is the command that it runs that fails. I assume that paketo is doing something that adds more layers to the image before it attempts to use it to build.

Motivations

CI build process does not work with the jammy full builder.

[1FastSTi]

We too are getting the same error

[jsandov]

@1FastSTi @outcomes-michael-pinnegar

I can build locally and in the CI using the following release:
paketobuildpacks/builder-jammy-full:0.3.387

The following releases will still produce the same error:

0.3.389
0.3.388

[sebastianplawner]

Same here. Had to fix the builder version to 0.3.387 or it won't work.

[taraszka]

Feel free to use https://github.com/migetapp/builder-ubuntu22-full, a squashed image that is binary compatible with paketobuildpacks/builder-jammy-full.
The builder is monitored via GitHub Actions, which checks for new releases of builder-jammy-full every 10 minutes and automatically triggers a new build whenever a new release is detected.