zaki work log

作業ログやら生活ログやらなんやら

トップ > GitLab > [OpenShift]プライベートに立てたGitLab CEのGitリポジトリからS2Iビルドする(OKD / CRC)

[OpenShift]プライベートに立てたGitLab CEのGitリポジトリからS2Iビルドする(OKD / CRC)

GitLab OpenShift OpenShift3 OKD 朝活 OpenShift4 tech

GitHubのリポジトリからだったらnew-appで一発で動くんだけど、プライベート環境のオレオレ証明書のGitリポジトリの場合について。

デプロイ対象のアプリ自体は以前Minishift用に作ったこれです。

qiita.com

ソース

github.com

このリポジトリのデータをそっくりそのままプライベートのGitLab CEのリポジトリにpushしてます。 GitLab CEについては直近のブログ記事のこの辺。

zaki-hmkc.hatenablog.com

環境

CentOS7にデプロイしたOKDのOpenShift 3.11で、masterノードで作業してます。

[zaki@okd-master ~]$ oc get node -o wide
NAME                      STATUS    ROLES          AGE       VERSION           INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION          CONTAINER-RUNTIME
okd-master.esxi.jp-z.jp   Ready     infra,master   22d       v1.11.0+d4cacc0   192.168.0.71   <none>        CentOS Linux 7 (Core)   3.10.0-862.el7.x86_64   docker://1.13.1
okd-node1.esxi.jp-z.jp    Ready     compute        22d       v1.11.0+d4cacc0   192.168.0.75   <none>        CentOS Linux 7 (Core)   3.10.0-862.el7.x86_64   docker://1.13.1
okd-node2.esxi.jp-z.jp    Ready     compute        22d       v1.11.0+d4cacc0   192.168.0.76   <none>        CentOS Linux 7 (Core)   3.10.0-862.el7.x86_64   docker://1.13.1
[zaki@okd-master ~]$ oc version
oc v3.11.0+62803d0-1
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://okd-master.esxi.jp-z.jp:8443
openshift v3.11.0+7876dd5-361
kubernetes v1.11.0+d4cacc0

new-app(設定なし)

DBは別途デプロイしておく(記事最後)

アプリをS2Iビルドするためのnew-appをプライベートのGitLab指定して実行。

$ oc new-app openshift/wildfly~https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2
--> Found image af69006 (7 months old) in image stream "openshift/wildfly" under tag "13.0" for "openshift/wildfly"

    WildFly 13.0.0.Final 
    -------------------- 
    Platform for building and running JEE applications on WildFly 13.0.0.Final

    Tags: builder, wildfly, wildfly13

    * A source build using source code from https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2 will be created
      * The resulting image will be pushed to image stream tag "javaee-memoapp2:latest"
      * Use 'start-build' to trigger a new build
    * This image will be deployed in deployment config "javaee-memoapp2"
    * Port 8080/tcp will be load balanced by service "javaee-memoapp2"
      * Other containers can access this service through the hostname "javaee-memoapp2"

--> Creating resources ...
    imagestream.image.openshift.io "javaee-memoapp2" created
    buildconfig.build.openshift.io "javaee-memoapp2" created
    deploymentconfig.apps.openshift.io "javaee-memoapp2" created
    service "javaee-memoapp2" created
--> Success
    Build scheduled, use 'oc logs -f bc/javaee-memoapp2' to track its progress.
    Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
     'oc expose svc/javaee-memoapp2' 
    Run 'oc status' to view your app.
[zaki@okd-master ~]$

一見成功しているように見えるけど…というかBuildConfigなどのリソース自体は作成される。

[zaki@okd-master ~]$ oc get bc
NAME              TYPE      FROM      LATEST
javaee-memoapp2   Source    Git       1
[zaki@okd-master ~]$ oc get build
NAME                TYPE      FROM      STATUS                       STARTED          DURATION
javaee-memoapp2-1   Source    Git       Failed (FetchSourceFailed)   31 seconds ago   20s

ただし失敗している。

[zaki@okd-master ~]$ oc logs build/javaee-memoapp2-1
Cloning "https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2" ...
error: fatal: unable to access 'https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2/': Could not resolve host: gitlab-ce.example.org; Unknown error

あら、DNS設定抜けてたｗ (想定外) ちょっとサーバを見直し。

[zaki@okd-master ~]$ ping gitlab-ce.example.org
PING gitlab-ce.example.org (192.168.0.21) 56(84) bytes of data.
64 bytes from 192.168.0.21 (192.168.0.21): icmp_seq=1 ttl=64 time=0.274 ms

--- gitlab-ce.example.org ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.274/0.274/0.274/0.000 ms

DNS設定を見直してリビルド

[zaki@okd-master ~]$ oc start-build javaee-memoapp2 
build.build.openshift.io/javaee-memoapp2-2 started

気を取り直して、

[zaki@okd-master ~]$ oc logs build/javaee-memoapp2-2
Cloning "https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2" ...
error: fatal: unable to access 'https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2/': Peer's certificate issuer has been marked as not trusted by the user.

証明書のエラーになる。

プライベートGitリポジトリの証明書を設定する

証明書のsecretを作る

ドキュメントはこの辺り:

access.redhat.com

[zaki@okd-master ~]$ oc create secret generic gitlabcert --from-file=ca.crt=cert/gitlab-ce.example.org.crt 
secret/gitlabcert created

ここではgitlabcertという名前のsecretリソースを作成。 証明書のcrtファイルは手元にcert/gitlab-ce.example.org.crtというファイルがあり、これはca.crtというキーは固定。

[zaki@okd-master ~]$ ll cert/gitlab-ce.example.org.crt 
-rw-r--r--. 1 zaki zaki 1822  2月  6 23:27 cert/gitlab-ce.example.org.crt
[zaki@okd-master ~]$ oc describe secret gitlabcert 
Name:         gitlabcert
Namespace:    memoapp
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
ca.crt:  1822 bytes

build configへ証明書secretの設定

作成したsecretを参照させる設定を追加する。

[zaki@okd-master ~]$ oc get bc javaee-memoapp2 -o yaml
:
:
spec:
:
:
  source:
    git:
      uri: https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2
    type: Git

ここ。

を

  source:
    git:
      uri: https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2
    type: Git
    sourceSecret:
      name: gitlabcert

こうする。

2/14追記: ↑はoc editを使う場合の変更ポイントだけど、oc set build-secretを使えばラク。

zaki-hmkc.hatenablog.com

再ビルド

[zaki@okd-master ~]$ oc start-build javaee-memoapp2 --follow 
build.build.openshift.io/javaee-memoapp2-3 started
Cloning "https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2" ...
error: RPC failed; result=22, HTTP code = 404
fatal: The remote end hung up unexpectedly

あら…

よく見たら、uriは

  source:
    git:
      uri: https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2.git
    type: Git
    sourceSecret:
      name: gitlabcert

でした。(.gitを付加)

[zaki@okd-master ~]$ oc start-build javaee-memoapp2 --follow 
build.build.openshift.io/javaee-memoapp2-4 started
Cloning "https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2.git" ...
        Commit: f0158568ff6e12d25011fb307836c40afa33d1c6 (update: readme)
        Author: zaki <zaki.hmkc@gmail.com>
        Date:   Mon Feb 11 18:57:25 2019 +0900
Using docker-registry.default.svc:5000/openshift/wildfly@sha256:73b9d5578eac447606e708a635bc0661755866162673a4c6e6d1119248d3c2c8 as the s2i builder image
Found pom.xml... attempting to build with 'mvn package -Popenshift -DskipTests -B -s /opt/app-root/src/.m2/settings.xml'
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
Maven home: /usr/local/apache-maven-3.5.4
Java version: 1.8.0_212, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "3.10.0-862.el7.x86_64", arch: "amd64", family: "unix"
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------< memoapp2:memoapp2 >--------------------------
[INFO] Building memoapp2 Maven Webapp 0.0.1-SNAPSHOT
[INFO] --------------------------------[ war ]---------------------------------
[INFO] Downloading from central: https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom
[INFO] Downloaded from central: https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom (8.1 kB at 9.2 kB/s)
[INFO] Downloading from central: https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-plugins/23/maven-plugins-23.pom
[INFO] Downloaded from central: https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-plugins/23/maven-plugins-23.pom (9.2 kB at 36 kB/s)
[INFO] Downloading from central: https://repo1.maven.org/maven2/org/apache/maven/maven-parent/22/maven-parent-22.pom
[INFO] Downloaded from central: https://repo1.maven.org/maven2/org/apache/maven/maven-parent/22/maven-parent-22.pom (30 kB at 82 kB/s)
[INFO] Downloading from central: https://repo1.maven.org/maven2/org/apache/apache/11/apache-11.pom

これでビルドが実行される。 JavaEEなのでmaven buildを勝手にやってくれてる。

:
:
Moving all ear artifacts from /opt/app-root/src/target directory into /wildfly/standalone/deployments for later deployment...
Moving all rar artifacts from /opt/app-root/src/target directory into /wildfly/standalone/deployments for later deployment...
Moving all jar artifacts from /opt/app-root/src/target directory into /wildfly/standalone/deployments for later deployment...
...done

Pushing image docker-registry.default.svc:5000/memoapp/javaee-memoapp2:latest ...
Pushed 0/13 layers, 0% complete
Pushed 1/13 layers, 8% complete
Pushed 2/13 layers, 16% complete
Pushed 3/13 layers, 23% complete
Pushed 4/13 layers, 33% complete
Pushed 5/13 layers, 40% complete
Pushed 6/13 layers, 49% complete
Pushed 7/13 layers, 68% complete
Pushed 8/13 layers, 72% complete
Pushed 9/13 layers, 76% complete
Pushed 10/13 layers, 88% complete
Pushed 11/13 layers, 100% complete
Pushed 12/13 layers, 100% complete
Pushed 13/13 layers, 100% complete
Push successful
[zaki@okd-master ~]$

しばらく待てば完了する。

[zaki@okd-master ~]$ oc get pod -l app=javaee-memoapp2
NAME                      READY     STATUS    RESTARTS   AGE
javaee-memoapp2-1-728xk   1/1       Running   0          39s

podも起動されている

new-appの時点で証明書指定

new-appのオプションに--source-secret=で証明書のsecret名を指定する。

[zaki@okd-master ~]$ oc new-app openshift/wildfly~https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2.git --source-secret=gitlabcert --name=memoapp
--> Found image af69006 (7 months old) in image stream "openshift/wildfly" under tag "13.0" for "openshift/wildfly"

    WildFly 13.0.0.Final 
    -------------------- 
    Platform for building and running JEE applications on WildFly 13.0.0.Final

    Tags: builder, wildfly, wildfly13

    * A source build using source code from https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2.git will be created
      * The resulting image will be pushed to image stream tag "memoapp:latest"
      * Use 'start-build' to trigger a new build
    * This image will be deployed in deployment config "memoapp"
    * Port 8080/tcp will be load balanced by service "memoapp"
      * Other containers can access this service through the hostname "memoapp"

--> Creating resources ...
    imagestream.image.openshift.io "memoapp" created
    buildconfig.build.openshift.io "memoapp" created
    deploymentconfig.apps.openshift.io "memoapp" created
    service "memoapp" created
--> Success
    Build scheduled, use 'oc logs -f bc/memoapp' to track its progress.
    Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
     'oc expose svc/memoapp' 
    Run 'oc status' to view your app.
[zaki@okd-master ~]$ 
[zaki@okd-master ~]$ oc get build -l app=memoapp
NAME        TYPE      FROM          STATUS    STARTED              DURATION
memoapp-1   Source    Git@f015856   Running   About a minute ago

ビルド動いてる。

[zaki@okd-master ~]$ oc get build -l app=memoapp
NAME        TYPE      FROM          STATUS     STARTED         DURATION
memoapp-1   Source    Git@f015856   Complete   2 minutes ago   1m38s
[zaki@okd-master ~]$ oc get pod -l app=memoapp
NAME              READY     STATUS    RESTARTS   AGE
memoapp-1-jt5q2   1/1       Running   0          1m

ビルド完了してデプロイされた。

OKD4がまだうまくデプロイできてないのでとりあえずOKD3で…
(ってよく考えたらCRC使って確認すればよかった)

CRC(OpenShift v4)の場合

2/7 朝 追記

全く同じ手順でビルドできます。
ただしopenshift/wildflyが(CRC 1.4だと)デフォルトで存在しないので、とりあえずopenshift/jboss-webserver31-tomcat8-openshiftをベースイメージに指定してビルド。(ビルドとデプロイができることを確認したのみ)

[zaki@codeready ~]$ oc new-app openshift/wildfly~https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2.git
error: unable to locate any images in image streams, local docker images with name "openshift/wildfly"
:
:

かわりにopenshift/jboss-webserver31-tomcat8-openshift:1.4を指定

[zaki@codeready ~]$ oc new-app openshift/jboss-webserver31-tomcat8-openshift:1.4~https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2.git
--> Found image 4b4b3e3 (8 weeks old) in image stream "openshift/jboss-webserver31-tomcat8-openshift" under tag "1.4" for "openshift/jboss-webserver31-tomcat8-openshift:1.4"

    JBoss Web Server 3.1 
    -------------------- 
    Platform for building and running web applications on JBoss Web Server 3.1 - Tomcat v8

    Tags: builder, java, tomcat8

    * A source build using source code from https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2.git will be created
      * The resulting image will be pushed to image stream tag "javaee-memoapp2:latest"
      * Use 'oc start-build' to trigger a new build
    * This image will be deployed in deployment config "javaee-memoapp2"
    * Ports 8080/tcp, 8443/tcp, 8778/tcp will be load balanced by service "javaee-memoapp2"
      * Other containers can access this service through the hostname "javaee-memoapp2"

--> Creating resources ...
    imagestream.image.openshift.io "javaee-memoapp2" created
    buildconfig.build.openshift.io "javaee-memoapp2" created
    deploymentconfig.apps.openshift.io "javaee-memoapp2" created
    service "javaee-memoapp2" created
--> Success
    Build scheduled, use 'oc logs -f bc/javaee-memoapp2' to track its progress.
    Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
     'oc expose svc/javaee-memoapp2' 
    Run 'oc status' to view your app.

証明書のsecret指定なしで実行すると

[zaki@codeready ~]$ oc logs build/javaee-memoapp2-1 
Cloning "https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2.git" ...
error: fatal: unable to access 'https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2.git/': Peer's certificate issuer has been marked as not trusted by the user.

同じようにエラー

証明書つくる

[zaki@codeready gitlab-cert]$ oc create secret generic gitlabcert --from-file=ca.crt=gitlab-ce.example.org.crt 
secret/gitlabcert created

証明書secret仕込み

[zaki@codeready gitlab-cert]$ oc edit bc javaee-memoapp2
buildconfig.build.openshift.io/javaee-memoapp2 edited
[zaki@codeready gitlab-cert]$ oc start-build javaee-memoapp2 --follow
build.build.openshift.io/javaee-memoapp2-2 started
Cloning "https://gitlab-ce.example.org:8443/zaki/javaee-memoapp2.git" ...
        Commit: f0158568ff6e12d25011fb307836c40afa33d1c6 (update: readme)
        Author: zaki <zaki.hmkc@gmail.com>
        Date:   Mon Feb 11 18:57:25 2019 +0900
Caching blobs under "/var/cache/blobs".
Getting image source signatures
:
:
:
Pushing image image-registry.openshift-image-registry.svc:5000/sample/javaee-memoapp2:latest ...
Getting image source signatures
Copying blob sha256:7d8a360d79ba467a388a156b152015ea2a534db3341e09011a0a659ffbb35784
Copying blob sha256:4abb233fe58adb32490c301b9a07ca6743c18c3a64e7ec5564129efaa5bd05fc
Copying blob sha256:04f8fdf93808f9bf0f7c23431334665ffeefd1a2fdab7ad09bf5802a6728d8e0
Copying blob sha256:487b0fd46e30b6edfa9bb916c4f0aa089de73a5c1086ccce0ef32a103919609b
Copying blob sha256:508f9bdf38c1f4ed86d55a8092ccffac87856d812fbc56240cde908912cf00f1
Copying config sha256:a2e58e31d38afcce501a858f39e59bfcfa4a51c816516f7ca4ab2071a092687d
Writing manifest to image destination
Storing signatures
Successfully pushed image-registry.openshift-image-registry.svc:5000/sample/javaee-memoapp2@sha256:b64954c1d941e3a0e730173914a84e2a97353603757216632eef9fdb744915e5
Push successful
[zaki@codeready gitlab-cert]$ 
[zaki@codeready gitlab-cert]$ oc get pod -l app=javaee-memoapp2
NAME                      READY   STATUS    RESTARTS   AGE
javaee-memoapp2-1-klhhz   1/1     Running   0          4m7s

環境

[zaki@codeready ~]$ crc version
crc version: 1.4.0+d5bb3a3
OpenShift version: 4.2.13 (embedded in binary)
[zaki@codeready ~]$ oc version
Client Version: v4.3.0
Server Version: 4.2.13
Kubernetes Version: v1.14.6+a8d983c

Quarkus入門しなきゃーと思いつつもう1年くらい経ってしまった…orz

quarkus.io

DBのデプロイ(参考)

PV用意してなかったのでephemeralで。

$ oc new-app mysql-ephemeral -p DATABASE_SERVICE_NAME=memoapp-db -p MYSQL_USER=memoapp -p MYSQL_PASSWORD=memoapp -p MYSQL_DATABASE=memoapp_db
--> Deploying template "openshift/mysql-ephemeral" to project memoapp

     MySQL (Ephemeral)
     ---------
     MySQL database service, without persistent storage. For more information about using this template, including OpenShift considerations, see https://github.com/sclorg/mysql-container/blob/master/5.7/root/usr/share/container-scripts/mysql/README.md.
     
     WARNING: Any data stored will be lost upon pod destruction. Only use this template for testing

     The following service(s) have been created in your project: memoapp-db.
     
            Username: memoapp
            Password: memoapp
       Database Name: memoapp_db
      Connection URL: mysql://memoapp-db:3306/
     
     For more information about using this template, including OpenShift considerations, see https://github.com/sclorg/mysql-container/blob/master/5.7/root/usr/share/container-scripts/mysql/README.md.

     * With parameters:
        * Memory Limit=512Mi
        * Namespace=openshift
        * Database Service Name=memoapp-db
        * MySQL Connection Username=memoapp
        * MySQL Connection Password=memoapp
        * MySQL root user Password=DDXJnYuU5c1bjadk # generated
        * MySQL Database Name=memoapp_db
        * Version of MySQL Image=5.7

--> Creating resources ...
    secret "memoapp-db" created
    service "memoapp-db" created
    deploymentconfig.apps.openshift.io "memoapp-db" created
--> Success
    Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
     'oc expose svc/memoapp-db' 
    Run 'oc status' to view your app.
[zaki@okd-master ~]$