Gitlab: build docker image in pipeline and use through registry.

A docker-boot job is added which builds the docker image and pushes it
to the registry, the other jobs running on the image from the
registry.

The boot job reuses the already pushed image if it exists and matches,
this is important to cut down its runtime.

Making a new image takes 30min for all the switches
https://gitlab.com/SkySkimmer/coq/-/jobs/66650546
For testing I removed all jobs except boot and main build, and after
that run I built only the standard switch.

Building 1 switch takes around 20min
https://gitlab.com/SkySkimmer/coq/-/jobs/66656942
When the registry is up to date it takes 2min
https://gitlab.com/SkySkimmer/coq/-/jobs/66658790
(I don't know about all switches but probably no more than 5min)

Each pipeline pushes the image to $CI_REGISTRY_IMAGE:$CI_PIPELINE_ID
which is eg registry.gitlab.com/skyskimmer/coq:21557176 and is what
the other jobs use to run themselves.

For caching it pulls from and pushes to a constant name, in this test
$CI_REGISTRY_IMAGE:$CACHEKEY. We might also want to pull from the main
coq repo to avoid redoing work.

The question of having 1 image or 1 image/switch remains open.

Using the gitlab registry doesn't really work for circle CI, so the
dockerhub account would have to remain (or circle could return to an
opam-boot job, or be removed from our CI). There are similar concerns
with travis if we want to move it to docker.

1 files changed, 16 insertions, 1 deletions

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 095099690..3ef592abc 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,6 +1,7 @@
-image: coqci/base:V2018-05-07-V2
+image: "$IMAGE"
 
 stages:
+  - docker
   - build
   - test
 
@@ -9,11 +10,25 @@ variables:
   # Format: $IMAGE-V$DATE [Cache is not used as of today but kept here
   # for reference]
   CACHEKEY: bionic_coq-V2018-05-07-V2
+  IMAGE: "$CI_REGISTRY_IMAGE:$CACHEKEY"
   # By default, jobs run in the base switch; override to select another switch
   OPAM_SWITCH: "base"
   # Used to select special compiler switches such as flambda, 32bits, etc...
   OPAM_VARIANT: ""
 
+docker-boot:
+  stage: docker
+  image: docker:stable
+  services:
+    - docker:dind
+  before_script: []
+  script:
+    - docker login -u gitlab-ci-token -p "$CI_JOB_TOKEN" "$CI_REGISTRY"
+    - cd dev/ci/docker/bionic_coq/
+    - if docker pull "$IMAGE"; then echo "Image prebuilt!"; exit 0; fi
+    - docker build -t "$IMAGE" .
+    - docker push "$IMAGE"
+
 before_script:
   - cat /proc/{cpu,mem}info || true
   - ls -a # figure out if artifacts are around