Skip to content

feat!(detector): timeout can be set, default is no timeout #2185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Apr 30, 2025
Merged

feat!(detector): timeout can be set, default is no timeout #2185

merged 3 commits into from
Apr 30, 2025

Conversation

MaineK00n
Copy link
Collaborator

@MaineK00n MaineK00n commented Apr 30, 2025 •
edited
Loading

If this Pull Request is work in progress, Add a prefix of “[WIP]” in the title.

What did you implement:

It is now possible to set the timeout for the entire request for each DB and for a single request.

Fixes #1695

Type of change

  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

before

$ vuls report --refresh-cve
[Apr 30 09:42:22]  INFO [localhost] vuls-0.30.0-358cbf59b8480330cebed319dee1bfc4c5704c7e-2025-03-18T06:42:29Z
...
[Apr 30 09:42:22]  INFO [localhost] Skip OVAL and Scan with gost alone.
[Apr 30 09:42:22]  INFO [localhost] localhost: 0 CVEs are detected with OVAL
[Apr 30 09:42:33]  WARN [localhost] Failed to HTTP GET. retrying in 349.858873ms seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/fixed-cves": read tcp 127.0.0.1:60838->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:162
[Apr 30 09:42:34]  WARN [localhost] Failed to HTTP GET. retrying in 733.697151ms seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves": read tcp 127.0.0.1:35692->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:162
[Apr 30 09:42:37]  WARN [localhost] Failed to HTTP GET. retrying in 362.707582ms seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves": read tcp 127.0.0.1:57228->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:162
[Apr 30 09:42:53]  WARN [localhost] Failed to HTTP GET. retrying in 523.661391ms seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/unfixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/unfixed-cves": read tcp 127.0.0.1:41338->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:162
[Apr 30 09:42:55]  WARN [localhost] Failed to HTTP GET. retrying in 291.867323ms seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/unfixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/unfixed-cves": read tcp 127.0.0.1:44874->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:162
[Apr 30 09:42:57]  WARN [localhost] Failed to HTTP GET. retrying in 710.885385ms seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/firefox/unfixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/firefox/unfixed-cves": read tcp 127.0.0.1:35386->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:162
[Apr 30 09:43:02]  WARN [localhost] Failed to HTTP GET. retrying in 679.438469ms seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/unfixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/unfixed-cves": read tcp 127.0.0.1:39344->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:162
[Apr 30 09:43:04]  WARN [localhost] Failed to HTTP GET. retrying in 592.415271ms seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/unfixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/unfixed-cves": read tcp 127.0.0.1:40358->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:162
[Apr 30 09:43:07]  INFO [localhost] localhost: 1412 CVEs are detected with gost
...

after

no timeout

$ cat config.toml
...
[gost]
type = "http"
url = "http://127.0.0.1:1325"
...

$ vuls report --refresh-cve
[Apr 30 10:09:08]  INFO [localhost] vuls-v0.30.0-build-20250430_100859_60d3afc
...
[Apr 30 10:09:08]  INFO [localhost] Skip OVAL and Scan with gost alone.
[Apr 30 10:09:08]  INFO [localhost] localhost: 0 CVEs are detected with OVAL
[Apr 30 10:09:33]  INFO [localhost] localhost: 1412 CVEs are detected with gost
...

TimeoutSec = 3

$ cat config.toml
...
[gost]
type = "http"
url = "http://127.0.0.1:1325"
timeoutSec = 3
...

$ vuls report --refresh-cve
[Apr 30 10:13:53]  INFO [localhost] vuls-v0.30.0-build-20250430_101344_60d3afc
...
[Apr 30 10:13:53]  INFO [localhost] Skip OVAL and Scan with gost alone.
[Apr 30 10:13:53]  INFO [localhost] localhost: 0 CVEs are detected with OVAL
[Apr 30 10:14:03] ERROR [localhost] Failed to detect Pkg CVE:
    github.com/future-architect/vuls/detector.Detect
        github.com/future-architect/vuls/detector/detector.go:54
  - Failed to detect CVE with gost:
    github.com/future-architect/vuls/detector.DetectPkgCves
        github.com/future-architect/vuls/detector/detector.go:341
  - Failed to detect CVEs with gost:
    github.com/future-architect/vuls/detector.detectPkgsCvesWithGost
        github.com/future-architect/vuls/detector/detector.go:593
  - Failed to detect CVEs. err:
    github.com/future-architect/vuls/gost.Ubuntu.DetectCVEs
        github.com/future-architect/vuls/gost/ubuntu.go:91
  - Failed to get fixed CVEs via HTTP. err:
    github.com/future-architect/vuls/gost.Ubuntu.detectCVEs
        github.com/future-architect/vuls/gost/ubuntu.go:110
  - Timeout Fetching Gost:
    github.com/future-architect/vuls/gost.getCvesWithFixStateViaHTTP
        github.com/future-architect/vuls/gost/util.go:143
...

TimeoutSecPerRequest = 1

$ cat config.toml
...
[gost]
type = "http"
url = "http://127.0.0.1:1325"
timeoutSecPerRequest = 1
...

$ vuls report --refresh-cve
[Apr 30 15:50:45]  INFO [localhost] vuls-v0.30.0-build-20250430_154216_d2347cb
...
[Apr 30 15:50:45]  INFO [localhost] Skip OVAL and Scan with gost alone.
[Apr 30 15:50:45]  INFO [localhost] localhost: 0 CVEs are detected with OVAL
[Apr 30 15:50:46]  WARN [localhost] Failed to HTTP GET. retrying in 0.263985 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/mysql-8.0/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/mysql-8.0/fixed-cves": read tcp 127.0.0.1:39478->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:46]  WARN [localhost] Failed to HTTP GET. retrying in 0.510949 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/qemu/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/qemu/fixed-cves": read tcp 127.0.0.1:39832->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:47]  WARN [localhost] Failed to HTTP GET. retrying in 0.353559 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/ffmpeg/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/ffmpeg/fixed-cves": read tcp 127.0.0.1:40242->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:47]  WARN [localhost] Failed to HTTP GET. retrying in 0.533346 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/vim/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/vim/fixed-cves": read tcp 127.0.0.1:40646->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:48]  WARN [localhost] Failed to HTTP GET. retrying in 0.501117 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/mysql-8.0/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/mysql-8.0/fixed-cves": read tcp 127.0.0.1:36746->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:48]  WARN [localhost] Failed to HTTP GET. retrying in 0.687424 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/ghostscript/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/ghostscript/fixed-cves": read tcp 127.0.0.1:36820->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:48]  WARN [localhost] Failed to HTTP GET. retrying in 0.350745 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/imagemagick/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/imagemagick/fixed-cves": read tcp 127.0.0.1:37056->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:48]  WARN [localhost] Failed to HTTP GET. retrying in 0.586388 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/ffmpeg/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/ffmpeg/fixed-cves": read tcp 127.0.0.1:37110->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:48]  WARN [localhost] Failed to HTTP GET. retrying in 0.468247 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/qemu/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/qemu/fixed-cves": read tcp 127.0.0.1:37126->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:48]  WARN [localhost] Failed to HTTP GET. retrying in 0.634093 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/vim/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/vim/fixed-cves": read tcp 127.0.0.1:37330->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:49]  WARN [localhost] Failed to HTTP GET. retrying in 0.706379 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/fixed-cves": read tcp 127.0.0.1:38156->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:49]  WARN [localhost] Failed to HTTP GET. retrying in 0.535043 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/imagemagick/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/imagemagick/fixed-cves": read tcp 127.0.0.1:38194->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:49]  WARN [localhost] Failed to HTTP GET. retrying in 1.029654 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/ghostscript/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/ghostscript/fixed-cves": read tcp 127.0.0.1:38252->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:50]  WARN [localhost] Failed to HTTP GET. retrying in 0.373260 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/webkit2gtk/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/webkit2gtk/fixed-cves": read tcp 127.0.0.1:39446->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:50]  WARN [localhost] Failed to HTTP GET. retrying in 0.652793 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/binutils/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/binutils/fixed-cves": read tcp 127.0.0.1:39452->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:50]  WARN [localhost] Failed to HTTP GET. retrying in 0.677239 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/openssl/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/openssl/fixed-cves": read tcp 127.0.0.1:39488->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:51]  WARN [localhost] Failed to HTTP GET. retrying in 0.714202 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/wireshark/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/wireshark/fixed-cves": read tcp 127.0.0.1:39558->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:51]  WARN [localhost] Failed to HTTP GET. retrying in 0.767817 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/fixed-cves": read tcp 127.0.0.1:39570->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:51]  WARN [localhost] Failed to HTTP GET. retrying in 0.361295 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/firefox/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/firefox/fixed-cves": read tcp 127.0.0.1:40248->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:52]  WARN [localhost] Failed to HTTP GET. retrying in 0.965213 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/webkit2gtk/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/webkit2gtk/fixed-cves": read tcp 127.0.0.1:40494->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:52]  WARN [localhost] Failed to HTTP GET. retrying in 0.746926 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/binutils/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/binutils/fixed-cves": read tcp 127.0.0.1:40710->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:52]  WARN [localhost] Failed to HTTP GET. retrying in 0.711244 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/exiv2/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/exiv2/fixed-cves": read tcp 127.0.0.1:40734->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:52]  WARN [localhost] Failed to HTTP GET. retrying in 0.924650 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/openssl/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/openssl/fixed-cves": read tcp 127.0.0.1:40746->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:52]  WARN [localhost] Failed to HTTP GET. retrying in 0.446524 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/wireshark/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/wireshark/fixed-cves": read tcp 127.0.0.1:40796->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:53]  WARN [localhost] Failed to HTTP GET. retrying in 0.483182 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/firefox/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/firefox/fixed-cves": read tcp 127.0.0.1:40856->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:54]  WARN [localhost] Failed to HTTP GET. retrying in 0.819730 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/exiv2/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/exiv2/fixed-cves": read tcp 127.0.0.1:41430->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:55]  WARN [localhost] Failed to HTTP GET. retrying in 0.494745 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/tiff/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/tiff/fixed-cves": read tcp 127.0.0.1:42114->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:55]  WARN [localhost] Failed to HTTP GET. retrying in 0.684500 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/mozjs91/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/mozjs91/fixed-cves": read tcp 127.0.0.1:42406->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:56]  WARN [localhost] Failed to HTTP GET. retrying in 0.532269 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/bind9/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/bind9/fixed-cves": read tcp 127.0.0.1:43478->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:56]  WARN [localhost] Failed to HTTP GET. retrying in 1.038578 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/tiff/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/tiff/fixed-cves": read tcp 127.0.0.1:43672->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:56]  WARN [localhost] Failed to HTTP GET. retrying in 0.395437 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves": read tcp 127.0.0.1:43960->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:57]  WARN [localhost] Failed to HTTP GET. retrying in 1.116910 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/mozjs91/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/mozjs91/fixed-cves": read tcp 127.0.0.1:44152->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:57]  WARN [localhost] Failed to HTTP GET. retrying in 0.366740 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/glibc/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/glibc/fixed-cves": read tcp 127.0.0.1:44156->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:57]  WARN [localhost] Failed to HTTP GET. retrying in 0.317072 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/xorg-server/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/xorg-server/fixed-cves": read tcp 127.0.0.1:44474->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:57]  WARN [localhost] Failed to HTTP GET. retrying in 0.323855 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves": read tcp 127.0.0.1:44554->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:57]  WARN [localhost] Failed to HTTP GET. retrying in 0.342929 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/krb5/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/krb5/fixed-cves": read tcp 127.0.0.1:44624->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:58]  WARN [localhost] Failed to HTTP GET. retrying in 0.687875 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/bind9/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/bind9/fixed-cves": read tcp 127.0.0.1:37230->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:58]  WARN [localhost] Failed to HTTP GET. retrying in 1.057722 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves": read tcp 127.0.0.1:37234->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:58]  WARN [localhost] Failed to HTTP GET. retrying in 0.583585 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/glibc/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/glibc/fixed-cves": read tcp 127.0.0.1:37272->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:58]  WARN [localhost] Failed to HTTP GET. retrying in 0.833720 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/xorg-server/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/xorg-server/fixed-cves": read tcp 127.0.0.1:37408->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:58]  WARN [localhost] Failed to HTTP GET. retrying in 0.550009 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/linux-hwe-6.8/fixed-cves": read tcp 127.0.0.1:37416->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:59]  WARN [localhost] Failed to HTTP GET. retrying in 0.593574 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/tcpdump/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/tcpdump/fixed-cves": read tcp 127.0.0.1:37808->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:50:59]  WARN [localhost] Failed to HTTP GET. retrying in 0.646464 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/radare2/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/radare2/fixed-cves": read tcp 127.0.0.1:37872->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:51:00]  WARN [localhost] Failed to HTTP GET. retrying in 1.009003 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/tcpdump/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/tcpdump/fixed-cves": read tcp 127.0.0.1:38702->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:51:01]  WARN [localhost] Failed to HTTP GET. retrying in 0.400232 seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/poppler/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/poppler/fixed-cves": read tcp 127.0.0.1:38770->127.0.0.1:1325: i/o timeout]:
    github.com/future-architect/vuls/gost.httpGet.func1
        github.com/future-architect/vuls/gost/util.go:168
[Apr 30 15:51:02] ERROR [localhost] Failed to detect Pkg CVE:
    github.com/future-architect/vuls/detector.Detect
        github.com/future-architect/vuls/detector/detector.go:54
  - Failed to detect CVE with gost:
    github.com/future-architect/vuls/detector.DetectPkgCves
        github.com/future-architect/vuls/detector/detector.go:341
  - Failed to detect CVEs with gost:
    github.com/future-architect/vuls/detector.detectPkgsCvesWithGost
        github.com/future-architect/vuls/detector/detector.go:593
  - Failed to detect CVEs. err:
    github.com/future-architect/vuls/gost.Ubuntu.DetectCVEs
        github.com/future-architect/vuls/gost/ubuntu.go:91
  - Failed to get fixed CVEs via HTTP. err:
    github.com/future-architect/vuls/gost.Ubuntu.detectCVEs
        github.com/future-architect/vuls/gost/ubuntu.go:110
  - Failed to fetch Gost. err: %!w([]error=[0xc002933bc0 0xc0021acde0 0xc0017f94d0 0xc001bba3f0 0xc00139c810 0xc0014b2de0 0xc0015caa20 0xc0018ca990 0xc0018cb380 0xc0018cb890 0xc001919ce0 0xc001aa3a70 0xc002554f30 0xc002c03f20 0xc0017f8570 0xc0015ca300 0xc001aa2d50 0xc00241cf30 0xc0022ce2a0]):
    github.com/future-architect/vuls/gost.getCvesWithFixStateViaHTTP
        github.com/future-architect/vuls/gost/util.go:147

TimeoutSec = 120, TimeoutSecPerRequest = 30

$ cat config.toml
...
[gost]
type = "http"
url = "http://127.0.0.1:1325"
timeoutSec = 120
timeoutSecPerRequest = 30
...

$ vuls report --refresh-cve
[Apr 30 10:19:49]  INFO [localhost] vuls-v0.30.0-build-20250430_101344_60d3afc
...
[Apr 30 10:19:49]  INFO [localhost] Skip OVAL and Scan with gost alone.
[Apr 30 10:19:49]  INFO [localhost] localhost: 0 CVEs are detected with OVAL
[Apr 30 10:20:20]  INFO [localhost] localhost: 1412 CVEs are detected with gost
...

Checklist:

You don't have to satisfy all of the following.

  • Write tests
  • Write documentation
  • Check that there aren't other open pull requests for the same issue/feature
  • Format your source code by make fmt
  • Pass the test by make test
  • Provide verification config / commands
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: YES

Reference

@MaineK00n MaineK00n self-assigned this Apr 30, 2025
@MaineK00n MaineK00n force-pushed the MaineK00n/report-timeout branch from ae2fc05 to 60d3afc Compare April 30, 2025 01:08
@MaineK00n MaineK00n marked this pull request as ready for review April 30, 2025 01:22
@MaineK00n MaineK00n requested a review from shino April 30, 2025 01:24
shino
shino reviewed Apr 30, 2025
View reviewed changes
@MaineK00n @shino
fix(detector/cve): fix typo
f7735e7 
Co-authored-by: Shunichi Shinohara <[email protected]>
@MaineK00n MaineK00n requested a review from shino April 30, 2025 06:32
@shino
Copy link
Collaborator

shino commented Apr 30, 2025

Not the scope of this PR, the log message makes me feels awkward.

[Apr 30 09:42:33]  WARN [localhost] Failed to HTTP GET. retrying in 349.858873ms seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/fixed-cves": read tcp 127.0.0.1:60838->127.0.0.1:1325: i/o timeout]:

Could you include this?

@@ -170,7 +170,7 @@ func httpGet(url string, req request, resChan chan<- response, errChan chan<- er
 		return nil
 	}
 	notify := func(err error, t time.Duration) {
-		logging.Log.Warnf("Failed to HTTP GET. retrying in %s seconds. err: %+v", t, err)
+		logging.Log.Warnf("Failed to HTTP GET. retrying in %s. err: %+v", t, err)
 	}
 	err := backoff.RetryNotify(f, backoff.NewExponentialBackOff(), notify)

@MaineK00n
Copy link
Collaborator Author

Not the scope of this PR, the log message makes me feels awkward.

[Apr 30 09:42:33]  WARN [localhost] Failed to HTTP GET. retrying in 349.858873ms seconds. err: HTTP GET error, url: http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/fixed-cves, resp: <nil>, err: [Get "http://127.0.0.1:1325/ubuntu/2204/pkgs/thunderbird/fixed-cves": read tcp 127.0.0.1:60838->127.0.0.1:1325: i/o timeout]:

Could you include this?

@@ -170,7 +170,7 @@ func httpGet(url string, req request, resChan chan<- response, errChan chan<- er
 		return nil
 	}
 	notify := func(err error, t time.Duration) {
-		logging.Log.Warnf("Failed to HTTP GET. retrying in %s seconds. err: %+v", t, err)
+		logging.Log.Warnf("Failed to HTTP GET. retrying in %s. err: %+v", t, err)
 	}
 	err := backoff.RetryNotify(f, backoff.NewExponentialBackOff(), notify)

fixed: d2347cb

shino
shino approved these changes Apr 30, 2025
View reviewed changes
Copy link
Collaborator

@shino shino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎊

@MaineK00n MaineK00n merged commit 4d85d36 into master Apr 30, 2025
7 checks passed
@MaineK00n MaineK00n deleted the MaineK00n/report-timeout branch April 30, 2025 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shino shino shino approved these changes

Assignees

@MaineK00n MaineK00n

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Performance issue
2 participants
@MaineK00n @shino