multi-arch alpine image support

[arm64b]

We need this definitely :-)

With that, you can:
'docker pull alpine:3.6' will get a x86_64 based docker image, while the same pull command will get an arm64-based docker image.

BTW,'libunwind-dev' pkg is missed from arm64 now...

[andyshinn]

I think this is in the works upstream. It may be something we can leverage when it is supported. But it first needs to be a Docker mechanism for detecting arch and switching the downloaded image on the fly.

According to the Fedora official image at https://github.com/docker-library/official-images/blob/master/library/fedora, this may be supported in the official library. I'm not quite sure what it means yet (if we would have to maintain all the arch or if they could be different repositories).

Either way, we also need some sort of build infra that can handle different arches. Right now we use CircleCI with is only amd64 AFAIK.

[tianon]

Couldn't we use the minirootfs tarballs upstream publishes? (see "MINI ROOT FILESYSTEM" on https://alpinelinux.org/downloads/)

That won't get us all the versions of Alpine supported here (nor edge), but it does cover at least 3.5 and 3.6 (for the supported arches of each).

As for official images, they can be in many repos, or a single repo, many branches or a single branch -- it's up to you. Here's the blurb I sent to @ncopa on the subject a little while ago (don't think anything was ever posted publicly, although I think we added you to the conversation at the time; probably got lost in the shuffle 😅):

we're finally officializing a lot of the multiarch work we've been doing, and as such I think we've finally got some decent examples that are worth looking at for how Alpine might be enabled for multiarch 🙂docker-library/official-images#3024 (ubuntu) and docker-library/official-images#3031 (debian) are our first two "base" images with multiarch enabled (no manifest lists yet, just official multiarch specification so we can build/push from a single source of truth -- manifest lists are on the TODO shortly though); I've set up https://github.com/tianon/docker-brew-alpine-multiarch as another example of how this could go down, using the minirootfs tarballs that have been officially released (with https://github.com/tianon/docker-brew-alpine-multiarch/blob/master/alpine being an example library file for it)

Since then, as you've noted, fedora has added this (which was very recent).

As for the Docker engine, it does have some preliminary multiarch support. Given a manifest list with proper platform specifications, it will pull an appropriate image for running based on the current engine's platform, and Swarm is even learning to schedule the appropriate images based on architecture. 😄

See docker-library/official-images#2289 for the tracking issue for official images + manifest lists. 👍

[andyshinn]

Can the official library just pull tarballs from the website or do they need to be in git?

[tianon]

Unfortunately they need to be in Git, although I'm happy to help automate that if you'd like!

[tianon]

I think we could use "date" and "time" from "latest-releases.yaml" to create reproducible commit hashes too.

[alexkreidler]

TLDR: I am working on a PR and would like input on the structure of our build artifacts

See my comments on #237 for initial thoughts and implementation details. I am trying to change as little code as possible but would like provide robust multi-arch support.

I've got the alpine-builder docker image able to pull the different architecture images supported by alpine into a rootfs.tar.xz.

Example

export RELEASE="v3.6"
export MIRROR="http://dl-cdn.alpinelinux.org/alpine"
export PACKAGES="alpine-baselayout,alpine-keys,apk-tools,libc-utils"
export ARCH="armhf"
export BUILD_OPTIONS=(-b -s -t UTC -r $RELEASE -m $MIRROR -p $PACKAGES -a $ARCH)
export TAGS=(alpine:3.6 alpine:latest)

However, this would require a separate folder and config for each architecture, like this:

 versions
    ├── gliderlabs-3.1-x86_64
    │   ├── Dockerfile
    │   └── options
    ...
    ├── library-3.6-x86-64
    │   ├── Dockerfile
    │   ├── options
    │   └── rootfs.tar.xz
    ├── library-3.6-aarch64
    │   ├── Dockerfile
    │   ├── options
    │   └── rootfs.tar.xz
    ...
    └── library-3.6-armhf
        ├── Dockerfile
        └── options

There would be an explosion in the number of folders, and it would be very difficult to maintain.
I am also opposed to generating multiple architectures in one run of the alpine-builder dockerfile.
One idea is to have another config variable, like TAGS, but for the architectures.

export RELEASE="v3.6"
export MIRROR="http://dl-cdn.alpinelinux.org/alpine"
export PACKAGES="alpine-baselayout,alpine-keys,apk-tools,libc-utils"
export ARCHS="x86_64,x86,armhf,aarch64"
export BUILD_OPTIONS=(-b -s -t UTC -r $RELEASE -m $MIRROR -p $PACKAGES)
export TAGS=(alpine:3.6 alpine:latest)

This would then just need an additional loop in the build.sh script to generate an image for each architecture of each release, and some additional code to generate the library spec file.

I'd love ideas on the best way to organize all this. Here is the debian structure for reference. I will update with a link to the WIP PR.

ping @andyshinn