can't find imported package github.com/lib/pq/hstore

[jlozano]

Hi all, I am trying to run go-fuzz-build on a local package, I have followed the instructions on the README and when I try to use the go-fuzz-build command I get the following error: can't find imported package github.com/lib/pq/hstore.

I then installed this package via go get github.com/lib/pq/hstore which seemed to work since the source files appeared in my go path.

However when I tried to use the go-fuzz-build command again I got the same error.

Any ideas what I should try? Or what may be going wrong?

I am on go version go1.6 darwin/amd64 and I have set GO15VENDOREXPERIMENT=1

Any help would be greatly appreciated, thank you!

[jlozano]

Turns out github.com/lib/pq is used elsewhere in my local code base, however it is not used in the package I am fuzzing and it is not used in any of the dependencies for the package I am fuzzing.

[dvyukov]

Can you provide a reproducer? Is it because of vendoring?

[msiebuhr]

I might've hit this same issue over at prometheus/prometheus#945.

Reproduction (untested, but the ideas should be obvious):

mkdir -p $GOROOT/src/github.com/prometheus
cd $GOROOT/src/github.com/prometheus
git clone github.com/msiebuhr/prometheus
cd prometheus
go get ./...
git checkout fuzz
go-fuzz-build -func FuzzParseMetric -o FuzzParseMetric.zip github.com/prometheus/prometheus/promql

One curious thing is that I get different import errors between runs. In particular it seems to pick one of three packages to fail with...

[dvyukov]

@msiebuhr These commands fail or me as:

/tmp/go-fuzz-build446977689/src/go.fuzz.main/main.go:10: undefined: promql.FuzzParseMetric

When I changed the command to:

$ go-fuzz-build -work -func FuzzParseMetric -o FuzzParseMetric.zip github.com/msiebuhr/prometheus/promql

It fails as:

# github.com/msiebuhr/prometheus/promql
./analyzer.go:83: cannot use a.Start.Add(-n.Offset - StalenessDelta) (type "github.com/msiebuhr/prometheus/vendor/github.com/prometheus/common/model".Time) as type "github.com/prometheus/prometheus/vendor/github.com/prometheus/common/model".Time in argument to a.Storage.MetricsForLabelMatchers

It seem that github.com/prometheus/prometheus is hardcoded in sources, and so your fork does not work. go build github.com/msiebuhr/prometheus/promql fails as well.

[msiebuhr]

@dvyukov It reads like you checked out my branch under github.com/msiebuhr/prometheus, when it should've been github.com/prometheus/prometheus.

(Alas, if only path-relative imports hadn't run into all sorts of practical realities :-/ )

As you already have it checked out, I guess mv $GOROOT/src/github.com/msiebuhr $GOROOT/src/github.com/prometheus could do the trick too.

[ryancox]

I'm hitting this as well; specifically with an app using a vendor directory. In looking at the src and comparing with my failures, it seems like it's failing in buildInstrumentedBinary. And that the deps array doesn't contain any of the applications dependencies.

In looking at a couple of the working directories I can see my app's source and the stdlib source explicitly added to the deps array. But no direct or transitive dependencies.

[dvyukov]

@msiebuhr Yeah, I tried to short-circuit your instructions. I was able to reproduce the failure. Works for me with 6174d61. Please retest.

I would strongly not suggest to do:

    _, err := ParseMetric(string(in))
    if err == nil {
        return fuzzInteresting
    }
    return fuzzDiscard

but instead do:

    _, err := ParseMetric(string(in))
    if err != nil {
        return fuzzMeh
    }
    return fuzzInteresting

It will be very difficult for the fuzzer to progress if it will constantly throw all progress away.

I have not looked at your package, but for parsing it's generally useful to do something along the lines of: parse, serialize, parse again, compare results of first and second parse.

[dvyukov]

@ryancox Please re-test on tip. @msiebuhr's reproducer is fixed now.

[msiebuhr]

@dvyukov It now works on Prometheus! And thanks for the tip on throwing too much stuff away.

[ryancox]

@dvyukov Works great. Thanks! Found a crasher right away: Return an error instead of nil