builder-jammy-full contains openssl-3.0.2 flagged with CVE-2022-1292 CVE-2022-2068 CVE-2024-5535

[patpatpat123]

Hello team,

Wanted to reach out with an issue.

We are building springboot application using graalvm native image.
The base image of choice is https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-3.4-Release-Notes#paketo-tiny-builder-for-building-oci-images paketobuildpacks/builder-jammy-java-tiny

As our application is running in production, we have many paid software which scans the content of the container to detect vulnerabilities.

All of our tools flags the following:

https://nvd.nist.gov/vuln/detail/CVE-2022-1292

https://nvd.nist.gov/vuln/detail/CVE-2022-2068

https://nvd.nist.gov/vuln/detail/CVE-2024-5535

We went to further confirm by trying to get inside the container.
Since builder-jammy-java-tiny does not allow user to "exec into" the container (by the way, this is great, thanks, it helped us pass our security review) we switched to builder-jammy-full to prove the point.

And indeed, we could see:

cnb@4e460311b36e:/workspace$ whoami
cnb


cnb@4e460311b36e:/workspace$ openssl version
OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)

Please note the console output, it is from inside the container, the container built with buildpack.

Furthermore, we can see the version is indeed openssl-3.0.2, which is aligned with the version flagged by all the vulnerability scanners.

This is preventing us from deploying to production

I understand we can just switch manually to another image of our choice, but we like using this builder.

Could you please help fix the vulnerability by bumping up the version of openssl to something newer and safer?

Thank you for your time.

[modulo11]

As all Paketo stacks are currently based on Ubuntu, please also check their security tracker. Often times Canonical fixes security issues "out of band" which is not known to all scanners.

• https://ubuntu.com/security/CVE-2022-1292: 22.04 LTS jammy, Fixed 3.0.2-0ubuntu1.1
• https://ubuntu.com/security/CVE-2022-2068: 22.04 LTS jammy, Fixed 3.0.2-0ubuntu1.5
• https://ubuntu.com/security/CVE-2024-5535: 22.04 LTS jammy Fixed 3.0.2-0ubuntu1.17

The builder currently ships version 3.0.2-0ubuntu1.18 of the openssl package.

[patpatpat123]

Hello @modulo11 , we called all the companies behind the tools showing our enterprise licenses.

All of them maintain the scan is not a false positive, but the vulnerability is legit.

Actually, the vulnerability also exists for builder-noble.

[loewenstein-sap]

Hi @patpatpat123,

as you can see in the jammy-tiny-stack-0.2.56-arm64-run-receipt.cyclonedx.json for the latest release of the Jammy Tiny stack, the OpenSSL version is 3.0.2-0ubuntu1.18, which includes fixes for all of the CVEs.

I would suggest to check with your vendors if they support recognizing Ubuntu package versions that ship additional patches.

[dmikusa]

I would also add that when you talk to vendors for scanning tools like these you need to ask them to explain to you why the tool believes that it's correct. Any reasonable tool for security scanning should be able to show you exactly why it thinks a vulnerability exists. When you know why, then you can make the determination yourself if it's correct or not.

[patpatpat123]

Hello @dmikusa , @modulo11 , @loewenstein-sap ,

Thank you for all the input provided.

After talking with the vendor company, it seems the flag is not a false positive, but a real one.

Here is the rationale.

It has been mentioned in this thread, I am quoting: "The builder currently ships version 3.0.2-0ubuntu1.18 of the package." and "3.0.2-0ubuntu1.18, which includes fixes for all of the CVEs".

This is correct, version 3.0.2-0ubuntu1.18 is not vulnerable, and it is actually not the culprit, you guys might not have looked at the correct thing to begin with.

After extracting the snippets of both package entries from the SBOM below, we can see TWO versions, and they both exist

Deb/dpkg package:

{
"bom-ref": "pkg:generic/[email protected]?package-id=fca182d086f8ec1f",
"type": "application",
"name": "openssl",
"version": "3.0.2",
"cpe": "cpe:2.3:a:openssl:openssl:3.0.2:*:*:*:*:*:*:*",
"purl": "pkg:generic/[email protected]",
"properties": [
{
"name": "syft:package:foundBy",
"value": "binary-classifier-cataloger"
},
{
"name": "syft:package:type",
"value": "binary"
},
{
"name": "syft:package:metadataType",
"value": "binary-signature"
},
{
"name": "syft:cpe23",
"value": "cpe:2.3:a:openssl:openssl:3.0.2:*:*:*:*:*:*:*"
},
{
"name": "syft:location:0:layerID",
"value": "sha256:d7926c8d51fbd4bb392cc7c36355a25352a9be11df8303426b82b6fff0261bd0"
},
{
"name": "syft:location:0:path",
"value": "/usr/bin/openssl"
}
]
},

Binary package: (but please do not look at me, I am ok, look at the one above)

{
"bom-ref": "pkg:deb/ubuntu/[email protected]?arch=arm64&distro=ubuntu-22.04&package-id=ebef1067338d401e",
"type": "library",
"publisher": "Ubuntu Developers <[[email protected]](mailto:[email protected])>",
"name": "openssl",
"version": "3.0.2-0ubuntu1.18",
"licenses": [
{
"license": {
"id": "Apache-2.0"
}
},
{
"license": {
"id": "GPL-1.0-only"
}
},
{
"license": {
"id": "GPL-1.0-or-later"
}
},
{
"license": {
"name": "Artistic"
}
}
],
"cpe": "cpe:2.3:a:openssl:openssl:3.0.2-0ubuntu1.18:*:*:*:*:*:*:*",
"purl": "pkg:deb/ubuntu/[email protected]?arch=arm64&distro=ubuntu-22.04",
"properties": [
{
"name": "syft:package:foundBy",
"value": "dpkg-db-cataloger"
},
{
"name": "syft:package:type",
"value": "deb"
},
{
"name": "syft:package:metadataType",
"value": "dpkg-db-entry"
},
{
"name": "syft:location:0:layerID",
"value": "sha256:d7926c8d51fbd4bb392cc7c36355a25352a9be11df8303426b82b6fff0261bd0"
},
{
"name": "syft:location:0:path",
"value": "/usr/share/doc/libssl3/copyright"
},
{
"name": "syft:location:1:layerID",
"value": "sha256:d7926c8d51fbd4bb392cc7c36355a25352a9be11df8303426b82b6fff0261bd0"
},
{
"name": "syft:location:1:path",
"value": "/var/lib/dpkg/status.d/openssl"
},
{
"name": "syft:metadata:installedSize",
"value": "1981"
}
]
},

From the two snippets here, we can see:

One version is safe, as you guys claim. (talking about 3.0.2-0ubuntu1.18)

But it seems there is something else, and this something else is indeed not safe. 3.0.2

Could you guys please help fix this vulnerability?

[loewenstein-sap]

@paketo-buildpacks/stacks-maintainers Could you comment on this please. I've checked and the recipe indeed shows entries from binary cataloger as well as dpkg cataloger and the former indeed has added the "root" version of openssl 3.0.2 without patch suffix.

[patpatpat123]

Could you please help on this @paketo-buildpacks/stacks-maintainers ?

[loewenstein-sap]

cc @paketo-buildpacks/stacks-contributors

[dmikusa]

We use a tool called syft to generate the files that you're referencing.

My read of that output is that it detected the openssl deb package installed, which it found via dpkg-db-cataloger. The other entry is detected using the binary-classifier-cataloger, which again just my understanding, means syft saw this binary file and read some information out of it. What you're seeing in that entry is the limited info it was able to pull out of the binary.

I agree that it's confusing that it's listed twice, and I don't know if that's just the way syft works or if that's an artifact of the way that we are running syft. That said, it really looks like it's talking about the same thing, so I disagree that this represents an actual security issue.

The first entry is clearly referencing the openssl package, and the second is referencing a file /usr/bin/openssl, which is a file that is installed by the package.

{
"name": "syft:location:0:path",
"value": "/usr/bin/openssl"
}

You can confirm that the package is installing that file by looking at the package contents here.

Further, they are both coming from the same layer, d7926c8d51fbd4bb392cc7c36355a25352a9be11df8303426b82b6fff0261bd0 so that gives some confidence that they are the same file and not two separate files in two separate layers.

If you wanted to dig into it more, you could grab the deb file from a trusted location, extract the files and compare the sha hash to what's in the tiny image. The hash files should be the same because we use the upstream packages.

[modulo11]

Maybe evaluating/limiting the individual catalogers could be a way to improve the SBOM quality.

[patpatpat123]

@dmikusa even from what you said, it is still being installed as a separate package since it is a dependency of the openssl 3.0.2-0ubuntu1.18 package and therefore is still analysed and treated as an individual package since that is how it exists in the image.

Layer hash only suggests that these were installed in the same image layer (which is expected because running a command like apt install openssl installed openssl package and openssl binary). Nothing more.

It is not a concrete proof the vulnerability is a false positive. The openssl binary does exist as a dependency of the openssl package installation so the vulnerability is fair.

With all the provided evidence from this thread, can buildpacks team consider to improve the image by dissipating this confusion in the SBOM?

[patpatpat123]

@dmikusa
To summarize the thread until this point:

Justifications to prove it is a false positive:

• both coming from the same layer, d7926c8d51fbd4bb392cc7c36355a25352a9be11df8303426b82b6fff0261bd0 so that gives some confidence that they are the same file and not two separate files in two separate layers.
• grab the deb file from a trusted location, extract the files and compare the sha hash to what's in the tiny image.
  => I went to do that. I see two different hashes

Justifications to prove this is not a false positive:

• there are two different openssl in this package, one not vulnerable 3.0.2-0ubuntu1.18, but one vulnerable 3.0.2
• the binary 3.0.2, vulnerable, exists in the container as version 3.0.2
• Saying the two are from the same layer, and therefore it is a false positive, is not a proof at all. just suggests that these were installed in the same image layer (which is expected because running a command like apt install openssl installed openssl package and openssl binary)

[dmikusa]

both coming from the same layer, d7926c8d51fbd4bb392cc7c36355a25352a9be11df8303426b82b6fff0261bd0 so that gives some confidence that they are the same file and not two separate files in two separate layers.

This means that there can be only one file because they are both in the same layer. You cannot have two files at exactly the same location in the same layer, a file system won't permit that. You can have two files at different paths or you can have two files at the same path in different layers, but not two different files at the same path and in the same layer.

So when you have a catalog entry from a deb package that installs /usr/bin/openssl and you have a catalog entry of a binary reported the same file at /usr/bin/openssl and they are in the same layer, then you have the same file. Because you can't have two different files at the same location in the same layer.

The binary in question, the one you claim is a problem, is listed at /usr/bin/openssl. The deb package that is installed which is OK writes the file /usr/bin/openssl. The layer into which the deb package is installed is the same layer listed by both entries in the SBOM.

In order for someone to change the /usr/bin/openssl file to be something other than what is installed by the deb, one would need to modify that file after the package is installed but before the command which installs the package completes because when it completes, docker build will write that layer and then the layer cannot be modified without changing it's hash. That's a very small window of time. I'm not saying it's impossible, but it's very unlikely.

grab the deb file from a trusted location, extract the files and compare the sha hash to what's in the tiny image. => I went to do that. I see two different hashes

Can you expand on this and show what commands you ran to verify? I would expect these to be the same, so long as all the versions match up.

there are two different openssl in this package, one not vulnerable 3.0.2-0ubuntu1.18, but one vulnerable 3.0.2

Can you clarify this? The report show one deb package and one file, not two different packages. You can tell this by looking at the cataloger.

the binary 3.0.2, vulnerable, exists in the container as version 3.0.2

Or the cataloger that reported this doesn't know the specific version because it simply doesn't have any more details available by scanning the binary itself.

Saying the two are from the same layer, and therefore it is a false positive, is not a proof at all. just suggests that these were installed in the same image layer (which is expected because running a command like apt install openssl installed openssl package and openssl binary)

The key is that they are both reporting the same file at the same location. If they were talking about two different files or two files at the same path in different layers, then I would agree with you. But you can't have two files different files at the same path in the same layer. Those are the same file.

[dmikusa]

My test:

Stock ubuntu:jammy container:

> docker run -it --rm ubuntu:jammy bash
root@5bb56e283fec:/# apt update
Get:1 http://ports.ubuntu.com/ubuntu-ports jammy InRelease [270 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports jammy-updates InRelease [128 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports jammy-backports InRelease [127 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports jammy-security InRelease [129 kB]
Get:5 http://ports.ubuntu.com/ubuntu-ports jammy/restricted arm64 Packages [24.2 kB]
Get:6 http://ports.ubuntu.com/ubuntu-ports jammy/multiverse arm64 Packages [224 kB]
Get:7 http://ports.ubuntu.com/ubuntu-ports jammy/main arm64 Packages [1758 kB]
Get:8 http://ports.ubuntu.com/ubuntu-ports jammy/universe arm64 Packages [17.2 MB]
Get:9 http://ports.ubuntu.com/ubuntu-ports jammy-updates/multiverse arm64 Packages [30.6 kB]
Get:10 http://ports.ubuntu.com/ubuntu-ports jammy-updates/restricted arm64 Packages [3013 kB]
Get:11 http://ports.ubuntu.com/ubuntu-ports jammy-updates/universe arm64 Packages [1477 kB]
Get:12 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 Packages [2571 kB]
Get:13 http://ports.ubuntu.com/ubuntu-ports jammy-backports/main arm64 Packages [81.0 kB]
Get:14 http://ports.ubuntu.com/ubuntu-ports jammy-backports/universe arm64 Packages [33.3 kB]
Get:15 http://ports.ubuntu.com/ubuntu-ports jammy-security/universe arm64 Packages [1190 kB]
Get:16 http://ports.ubuntu.com/ubuntu-ports jammy-security/restricted arm64 Packages [2882 kB]
Get:17 http://ports.ubuntu.com/ubuntu-ports jammy-security/multiverse arm64 Packages [24.2 kB]
Get:18 http://ports.ubuntu.com/ubuntu-ports jammy-security/main arm64 Packages [2275 kB]
Fetched 33.4 MB in 4s (8511 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
7 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@5bb56e283fec:/# apt install openssl
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
  ca-certificates
The following NEW packages will be installed:
  openssl
0 upgraded, 1 newly installed, 0 to remove and 7 not upgraded.
Need to get 1163 kB of archives.
After this operation, 2029 kB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 openssl arm64 3.0.2-0ubuntu1.18 [1163 kB]
Fetched 1163 kB in 1s (1359 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package openssl.
(Reading database ... 4387 files and directories currently installed.)
Preparing to unpack .../openssl_3.0.2-0ubuntu1.18_arm64.deb ...
Unpacking openssl (3.0.2-0ubuntu1.18) ...
Setting up openssl (3.0.2-0ubuntu1.18) ...
root@5bb56e283fec:/# sha256sum /usr/bin/openssl
ade5564ee9c8ea2eb324f0d4b7a8bcb8567d8bb8bf9c00f2b1e26fc4dc530c85  /usr/bin/openssl

Latest paketo jammy tiny container:

> docker run -it --rm paketobuildpacks/build-jammy-tiny:latest sh
$ shasum
^C
$ ls
bin  bindings  boot  dev  etc  home  lib  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
$ shasum -a 256 /usr/bin/openssl
ade5564ee9c8ea2eb324f0d4b7a8bcb8567d8bb8bf9c00f2b1e26fc4dc530c85  /usr/bin/openssl

For the base stack, we do not have arm64 containers so the hash is different. Note how I'm forcing amd64 for the ubuntu:jammy image.

> docker run -it --rm --platform linux/amd64 ubuntu:jammy bash
Unable to find image 'ubuntu:jammy' locally
jammy: Pulling from library/ubuntu
9cb31e2e37ea: Pull complete
Digest: sha256:ed1544e454989078f5dec1bfdabd8c5cc9c48e0705d07b678ab6ae3fb61952d2
Status: Downloaded newer image for ubuntu:jammy
root@14678f867fc2:/# apt update
Get:1 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Get:8 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2910 kB]
Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [3748 kB]
Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [53.3 kB]
Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1523 kB]
Get:13 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [81.4 kB]
Get:14 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [35.2 kB]
Get:15 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [45.2 kB]
Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1230 kB]
Get:17 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2606 kB]
Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [3606 kB]
Fetched 36.2 MB in 4s (8053 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
7 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@14678f867fc2:/# apt install openssl
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
  ca-certificates
The following NEW packages will be installed:
  openssl
0 upgraded, 1 newly installed, 0 to remove and 7 not upgraded.
Need to get 1184 kB of archives.
After this operation, 2102 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 openssl amd64 3.0.2-0ubuntu1.18 [1184 kB]
Fetched 1184 kB in 0s (3589 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package openssl.
(Reading database ... 4393 files and directories currently installed.)
Preparing to unpack .../openssl_3.0.2-0ubuntu1.18_amd64.deb ...
Unpacking openssl (3.0.2-0ubuntu1.18) ...
Setting up openssl (3.0.2-0ubuntu1.18) ...
root@14678f867fc2:/# sha256sum /usr/bin/openssl
4bfdc44f0870920e5bb599d3dadaaadd612cf3f59e93e5f5dbcf1bfa852e18fc  /usr/bin/openssl

and paketo jammy base container:

> docker run -it --rm paketobuildpacks/build-jammy-base:latest sh
Unable to find image 'paketobuildpacks/build-jammy-base:latest' locally
latest: Pulling from paketobuildpacks/build-jammy-base
aaf908379da9: Pull complete
c9d6f8e95f9f: Pull complete
df96e507d6e4: Pull complete
Digest: sha256:7fa05a1ba4516088ad1a2d2a984399d3f02ba78a3dece91638c21b7c0e950d78
Status: Downloaded newer image for paketobuildpacks/build-jammy-base:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
$ sha256sum /usr/bin/openssl
4bfdc44f0870920e5bb599d3dadaaadd612cf3f59e93e5f5dbcf1bfa852e18fc  /usr/bin/openssl

In both tests, the files match exactly, so I see no evidence of them being tampered with.

[patpatpat123]

Hey @dmikusa ,

Thank you for your time on this.

Can this clarify?
This is showing that the openssl package contains the openssl binary, so these are 2 different things

apt show openssl

Package: openssl
Status: install ok installed
Priority: optional
Section: utils
Installed-Size: 2053
Maintainer: Ubuntu Developers <[[email protected]](mailto:[email protected])>
Architecture: amd64
Multi-Arch: foreign
Version: 3.0.2-0ubuntu1.18
Depends: libc6 (>= 2.34), libssl3 (>= 3.0.2-0ubuntu1.2)
Suggests: ca-certificates
Conffiles:
 /etc/ssl/openssl.cnf 6b4a72a4ce84bab35d884e536295e14f
Description: Secure Sockets Layer toolkit - cryptographic utility
 This package is part of the OpenSSL project's implementation of the SSL
 and TLS cryptographic protocols for secure communication over the
 Internet.
 .
It contains the general-purpose command line binary /usr/bin/openssl,
 useful for cryptographic operations such as:
  * creating RSA, DH, and DSA key parameters;
  * creating X.509 certificates, CSRs, and CRLs;
  * calculating message digests;
  * encrypting and decrypting with ciphers;
  * testing SSL/TLS clients and servers;
  * handling S/MIME signed or encrypted mail.
Homepage: https://www.openssl.org/
Original-Maintainer: Debian OpenSSL Team <[[email protected]](mailto:[email protected])>