clarify Go support policy for secondary ports

[ianlancetaylor]

Background

This proposal started out as a GitHub discussionn at #53060.

Go supports a number of different GOOS/GOARCH targets. We've defined a policy for adding new ports, described at https://go.dev/wiki/PortingPolicy.

Ports are divided into first class ports and secondary ports. The current first class ports are:

• darwin/amd64
• darwin/arm64
• linux/386
• linux/amd64
• linux/arm
• linux/arm64
• windows/386
• windows/amd64

The current secondary ports are:

• aix/ppc64
• dragonfly/amd64
• freebsd/386
• freebsd/amd64
• freebsd/arm
• freebsd/arm64
• illumos/amd64
• linux/ppc64
• linux/ppc64le
• linux/mips
• linux/mipsle
• linux/mips64
• linux/mips64le
• linux/riscv64
• linux/s390x
• android/386
• android/amd64
• android/arm
• android/arm64
• ios/arm64
• ios/amd64
• js/wasm
• netbsd/386
• netbsd/amd64
• netbsd/arm
• netbsd/arm64
• openbsd/386
• openbsd/amd64
• openbsd/arm
• openbsd/arm64
• openbsd/mips64
• plan9/386
• plan9/amd64
• plan9/arm
• solaris/amd64
• windows/arm
• windows/arm64

The core Go team maintains the first class ports. It is less clear how the secondary ports are handled.

The existing porting policy says:

• Every secondary port must have a maintainer who will make required updates in a timely manner.
• Every secondary port must have a builder, and a person who maintains that builder.
• If the builder for a port is failing for more than two weeks, a new maintainer is needed.
• If a builder fails for more than four weeks or is failing at the time of a release freeze, and a new maintainer cannot be found, the port will be removed from the tree.

However, in practice we do not follow those rules.

• It has not been clear who the secondary port maintainers are, and there was no mechanism for adding or removing maintainers; this was recently improved by creating the @golang/port-maintainers GitHub team and subteams.
• When a change requires an update to the port, it is the person making the change who takes responsibility for updating the port, not the port maintainer.
• Maintainer responsibilities are in general unclear.
• Several secondary port builders are maintained by the Go team.
• We do not remove ports from the tree if they are failing for four weeks (we do remove ports if the OS is no longer supported, such as darwin/386).

The effect is that the work required to maintain secondary ports falls on people who are not familiar with those ports. This was not the goal of the porting policy, and it tends to slow down overall development of the core Go systems and discourages the adoption of new secondary ports.

We propose both loosening and tightening the current porting policy to address these concerns.

Proposal

• The Go team will stop publishing binaries for secondary ports.
  • The Go team currently publishes binaries for the following secondary ports, which we would stop doing if we adopt this part of the proposal:
    • freebsd/386
    • freebsd/amd64
    • linux/ppc64le
    • linux/s390x
    • windows/arm64
  • We encourage port maintainers to publish them instead; we can link to download pages from https://go.dev/dl as appropriate.
  • We will investigate building binary releases by cross-compiling to secondary ports from a first-class port. This may be feasible by doing pure Go builds, while leaving any cgo builds to be on the user's machine using the user's C compiler.
  • Note: when and if reliable testing hardware is available, the Go team is likely to promote windows/arm64 to be a first class port.
• Require at least two maintainers for each secondary port.
  • If maintainers resign or become unreachable and can't be replaced, we mark the port as broken; see below.
  • This means that for some existing secondary ports we will have to find additional maintainers. As I write this I think that additional maintainers are needed for android, dragonfly, illumos, ios, js/wasm, openbsd, and solaris.
• In general developers will not be required to make their code work on every secondary port.
  • They are encouraged to do so if it is straightforward.
  • If not, they are encouraged to notify port maintainers about pending requirements as soon as feasible, and to work with the port maintainers to get the ports working again.
  • That said, ultimately the port maintainers are responsible for keeping their ports working.
• Notify maintainers when a port breaks.
  • Open an issue, CC'ing maintainers
    • Perhaps also send an e-mail to golang-dev.
  • We can do this preemptively if we know that some change is going to break a secondary port.
  • Ideally this is an automated system, but for now it may be manual.
    • For example, perhaps the system that creates https://build.golang.org/ can send e-mails about all failures to a newly created list. Maintainers can filter that list for the ports that they care about.
    • Currently maintainers can track build dashboard triage log #52653 to see problems.
• Clarify maintainer responsibilities on the PortingPolicy wiki page.
  • A GOOS maintainer is responsible for any _GOOS files, any files with just that build tag, and blocks of code guarded by if runtime.GOOS == "mygoos".
  • Similarly for a GOARCH maintainer.
  • If technically possible, GOOS/GOARCH maintainers who are not already approvers should be given the right to +2 changes to files for which they are responsible.
    • These CLs will still require a +1 review for style and sanity but not for detailed correctness; we will defer to the maintainers for that.
    • Note that with two maintainers one of them can +2 changes by the other.
• We will introduce a new concept: the broken port.
  • The existing policy says that problems must be fixed within 4 weeks, which is not realistic; people are busy.
  • If a port stops working, including the case where a builder stops working, we can decide to mark the port as broken.
    • Or in some cases we can roll back the change that broke it; this is a judgement call.
  • In general, a port can be considered broken if its builder has failed multiple times in a development cycle with a failure mode that does not occur for first class ports, and that failure mode is not believed to have been fixed or suppressed by a change in either a Go repository or the builder's configuration, and maintainers are not actively working on a solution.
  • Any approver can declare that a port that meets these criteria is broken.
  • The list of broken ports will be maintained in cmd/dist and will appear in the release notes if broken at release time.
  • Attempting to build a broken port will fail unless make.bash or go tool dist is invoked with a new option -force.
  • If a port is broken in release 1.N, then the core Go team can choose to remove the port from release 1.N+1.
    • This is not obligatory and will depend on whether anybody is willing and able to maintain the port going forward.
  • We currently have a list of incomplete ports in cmd/dist/build.go; these should probably be treated as broken ports.
    • Currently the only entry on that list is linux/sparc64.
  • The goal here is not to get ports out of the tree; if people are actively working on the port they should have as much as latitude as possible to fix it.
• The default set of trybots will change to only cover first class ports.
  • The https://build.golang.org/ page will continue to show all ports.

Discussion

This is not intended to be a big change to the current process. However, it is intended to be a change. It is intended to take some of the porting load off of the core Go team, while making it easier for port maintainers to make changes. It is intended to make it easier to add new ports to the tree.

In the long run it would be good to support out of tree ports. However, that requires a bunch of technical work, and there is no design for it.

[erifan]

In general developers will not be required to make their code work on every secondary port.

Does this mean that we can only consider the first port when adding new features in the future, such as regabi.

And will we move a lot of secondary port related code out of the master branch, so that our code can work without regard to these ports.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[ianlancetaylor]

@erifan

Does this mean that we can only consider the first port when adding new features in the future, such as regabi.

I think it depends on what you mean by "consider." I think an implementation for anything that covers all GOARCH values but differs for each one must as least consider whether and how it can be implemented for each GOARCH. We shouldn't add features that can never work for some GOARCH. And the initial implementation should not break any other GOARCH. But it's OK to implement something like regabi one GOARCH at a time, as indeed is being done for regabi.

And will we move a lot of secondary port related code out of the master branch, so that our code can work without regard to these ports.

I am not proposing that at all. That would be what I call an "out of tree port." I think it would be great to support that, but that is very hard, and it is not this proposal.

[erifan]

Thanks @ianlancetaylor , I see. This sentence "This is not intended to be a big change to the current process. However, it is intended to be a change" is very accurate.

[beoran]

@ianlancetaylor

The core problem with porting Go to other platforms, and making out of tree ports now is that the go runtime, standard library and compiler are not very modular when it comes to operating system and architecture support. Now, packes have files for all platforms confounded are in the same package using build tags. I think this should be refactored and split up into packages per operating system and/or architecture.

One should be able to write a platform support for the Go compiler, runtime and std lib just by writing a module which the compiler , etc. could then import.

If course this would involve a serious refactoring of Go, but I think it will be for the better, and encourage third party ports.

[ianlancetaylor]

@beoran Thanks. I agree with all of that. But this is not that issue. The changes proposed here should not have to wait for the changes that you suggest. The changes you suggest will take a long time to design and implement, and nobody is working on them.

[beoran]

@ianlancetaylor Well I wanted to bring it up but I agree it is a separate issue in the end.

[aclements]

@beoran , we are making baby steps toward better modularization in the runtime by moving the per-OS/arch syscall layer into runtime/internal/syscall and reorganizing it to have significantly less per-arch code. We're part way through migrating the Linux arches (we've migrated the raw syscall functions for all Linux arches, but not yet the Go interface). It's a small step, but should help some.

[rsc]

I think the discussion helped a lot with addressing most of people's concerns, but I'm still a bit surprised about how little discussion is happening here. Does anyone have any remaining objections that we should discuss?

[qmuntal]

Note: when and if reliable testing hardware is available, the Go team is likely to promote windows/arm64 to be a first class port.

Microsoft dev here. We would like to help promoting windows/arm64 to be a first-class port. If it is just a problem of having reliable hardware, we can discuss providing reliable windows/arm64 hosts running in our own infrastructure. Would that be enough? If not, what else would be missing?

[ianlancetaylor]

CC @golang/port-maintainers

[ianlancetaylor]

CC @golang/release See comment by @qmuntal above regarding reliable windows/arm64 hardware. That should probably be discussed on a separate issue. Thanks.