add an "export" command for libraries

[mvdan]

In the context of #276 (comment), I've been thinking that there are scenarios where one might want to obfuscate only some packages: when publishing or packaging an obfuscated library, to be consumed as regular Go packages via source code.

We do sort of support this already, via something like:

GOPRIVATE=my.corp/some/library garble -debugdir=out build my.corp/some/library/...

Then, the obfuscated source code will be under out/my.corp/some/library.

I think this use case is valid, given that it's not possible (or at least not easy) to distribute Go libraries without distributing source code too. One such example is unidoc, where its libraries like unipdf are obfuscated then pushed to GitHub.

We already have much of the machinery here, and garble is pretty advanced as a Go obfuscator, so I think it makes sense to take it one step further and make this easier to do. For example, via:

garble export out my.corp/some/library/...

Some key differences compared to how we currently obfuscate code:

• Import paths should remain untouched, for the sake of keeping imports working.
• Exported names should also remain untouched, such as types and functions.
• Godoc comments of exported names should be left in place.
• License comments should be left in place.

Internal packages such as my.corp/some/library/internal/foo would still be fully obfuscated, including changing their API names and import paths. That could be a way to tell what packages must remain importable and usable.

Otherwise, I think obfuscation could stay the same: stripping unexported names, position information, most comments, etc.

We might have to tweak our obfuscator to collapse newlines, too - right now, it does that via /*line comments for the compiler, but newlines in the printed source code remain, since those don't affect binaries. They affect published source code, though - and it's also likely that those compiler directives shouldn't be present in this new mode.

[mvdan]

It's unclear to me if this means we should keep GOPRIVATE or GOGARBLE support around.

On one hand, it makes sense to keep GOGARBLE as a configuration option, because garble export out my.corp/some/library/... is implicitly using GOGARBLE=my.corp/some/library. We'd need to keep support for only obfuscating some packages but not others, too.

On the other hand, would anyone ever set GOGARBLE directly? When building manually, one would (presumably) always want to obfuscate all packages. When exporting a library like the example above, one would just want to obfuscate the library's packages.

One possible use case for GOGARBLE would be to emulate the garble export configuration for other commands, like GOGARBLE=my.corp/some/library go test my.corp/some/library/.... However, note that obfuscation of binaries would be different, so this wouldn't be testing the same code that got exported. Perhaps we could add something like garble export-test in the future.

Right now, I'm still leaning towards phasing out GOGARBLE/GOPRIVATE support.

[mvdan]

Also, I think this command should be akin to garble build, in that packages should be built as they are obfuscated to ensure they haven't been broken in ways that trigger the parser or typechecker. We could later consider an option to skip building and just obfuscate.

[lu4p]

I think unidocs obfuscation is cute, I might try to reverse it tomorrow.

[mvdan]

I just realised what is the biggest hurdle with garble export: build tags. Right now, the way we obfuscate code relies on go/types and go/ssa, both of which "compile" each package with a set of types given some specific build tags.

But when exporting, we want to obfuscate all files - no matter the GOOS, GOARCH, or any other build tags. If a package has foo_linux.go and foo_windows.go, the user likely wants both to be obfuscated and exported.

I can think of two options:

1. Only obfuscate and export for one set of build tags, e.g. the current GOOS and GOARCH. This is easy, but rather limiting.

2. Have the user give us a list of build tag sets, e.g. garble export -tags="linux,amd64 windows,amd64 darwin,arm64", and we perform the obfuscation for each of them - joining the results at the end so that all files are obfuscated. Note that simply deduplicating obfuscated files won't be enough, since different build tags will currently result in different obfuscation. We would have to teach garble to obfuscate the same way for each of the build tag sets. This would also slow down the export, as we would do one obfuscated build per tag set.

A third option would be to obfuscate in some way that doesn't rely on build tags. But this would require giving up go/types and go/ssa, and effectively rewriting garble to be significantly less powerful.

[pagran]

2 - what if we consider each tag as a unique build and put all files obfuscated with this tag under this tag as well? This would highly increase the total size (and obfuscation time), but make the resulting package universal and with unique obfuscation for each tag (tags combination?).

[mvdan]

This could multiply the size of Go modules though, so I'd rather avoid it unless there are significant benefits. I don't see any significant benefits, given that I expect that most users would want to publish all the "build tag variants" alongside each other. Spotting the common bits between them wouldn't be that hard.