Развёртывание кластера на основе OKD/OCP

Для развёртывания отказоустойчивого кластера на основе OKD/OCP выполните следующие действия на хосте srv:

  1. Установите стабильные версии 4.6 oc client и openshift-installer (OKD) с помощью команд, выполняемых в веб-интерфейсе WUI или в консоли через подключение по SSH:

VERSION=4.6.0-0.okd-2021-02-14-205305
mkdir -p ~/openshift/install 
cd ~/openshift
wget https://github.com/openshift/okd/releases/download/$VERSION/openshift-client-linux-$VERSION.tar.gz
wget https://github.com/openshift/okd/releases/download/$VERSION/openshift-install-linux-$VERSION.tar.gz
tar -zxvf  openshift-client-linux-$VERSION.tar.gz
tar -zxvf openshift-install-linux-$VERSION.tar.gz
mv -f kubectl oc openshift-install /usr/local/bin/

  1. Сгенерируйте SSH-ключи с помощью команды:

ssh-keygen -t ed25519 -N ''

  1. Создайте манифесты с помощью команд:

cd /root/openshift/install
cat << EOF > ./install-config.yaml 
apiVersion: v1
baseDomain: okd.lan
metadata: 
name: prod
compute:
  - hyperthreading: Enabled 
name: worker 
replicas: 0
controlPlane: 
hyperthreading: Enabled 
name: master 
replicas: 3
networking: 
clusterNetwork: 
  - cidr: 10.128.0.0/14    
hostPrefix: 23  
machineNetwork: 
  - cidr: 192.168.0.0/24 
networkType: OVNKubernetes 
serviceNetwork:  
- 172.30.0.0/16
platform: 
none: {}
fips: false
pullSecret: '{"auths":{"fake":{"auth": "bar"}}}' 
sshKey: '<публичный SSH-ключ>'

EOF
openshift-install create manifests
rm -f openshift/99_openshift-cluster-api_master-machines-*.yaml openshift/99_openshift-cluster-api_worker-machineset-*.yaml
sed -i 's/mastersSchedulable: true/mastersSchedulable: false/' manifests/cluster-scheduler-02-config.yml
openshift-install create ignition-configs
cat << EOF > ./append-bootstrap.ign 
{
"ignition": {   
"config": {   
"merge": [
        {
        "source": "http://192.168.0.2:8080/openshift/bootstrap.ign" 
        }   
]   
},   
"version": "3.1.0"
 }
}
EOF
cat << EOF > ./append-master.ign 
{
"ignition": {   
"config": {   
"merge": [
        {
        "source": "http://192.168.0.2:8080/openshift/master.ign" 
        }   
]   
},   
"version": "3.1.0"
 }
}
EOF
cat << EOF > ./append-worker.ign 
{
"ignition": {   
"config": {   
"merge": [
        {
        "source": "http://192.168.0.2:8080/openshift/worker.ign" 
        }   
]   
},   
"version": "3.1.0"
 }
}
EOF
base64 -w0 append-bootstrap.ign > append-bootstrap.64
base64 -w0 append-master.ign > append-master.64
base64 -w0 append-worker.ign > append-worker.64
mkdir /var/www/html/openshift
chown -R apache:apache /var/www/html/openshift
cp ./*.ign /var/www/html/openshift/
chown -R apache:apache /var/www/html/openshift/*
chmod 644 /var/www/html/openshift/*

В подстановке <публичный SSH-ключ> задайте сгенерированный SSH-ключ, содержащийся в файле с расширением *.pub.

  1. Создайте узлы кластера с помощью vCenter (WUI среды виртуализации):

    1. Выберите установленный шаблон VMware и склонируйте виртуальную машину с помощью команды «Clone > Clone to Virtual Machine».

    2. Укажите наименование виртуальной машины в соответствии с системными требованиями.

    3. Выберите «Customize this virtual machine`s hardware».

    4. Укажите необходимое количество CPU и RAM, подключите диски в соответствии с системными требованиями.

    5. Настройте коммутацию сетевых интерфейсов виртуальной машины в соответствии со схемой взаимодействия компонентов кластера.

    6. Отредактируйте дополнительные параметры виртуальной машины с помощью команды «VM Options > Advanced > Edit Configuration». Укажите для каждой виртуальной машины соответствующее значение:

Примечание. Наименования и IP-адреса хостов указаны для примера развёртывания кластера на основе OKD/OCP и могут отличаться.

Примечание. Значение для конкретного типа узла содержится на хосте srv после создания манифестов.

    1. Создайте виртуальную машину.

    2. Повторите вышеперечисленные действия для каждой виртуальной машины.

  1. Включите виртуальные машины в следующей последовательности:

Для получения подробной информации о виртуальных машинах обратитесь к документации OKD.

  1. Выполните следующие команды для наблюдения за процессом установки кластера на хосте srv:

export KUBECONFIG=~/openshift/install/auth/kubeconfig
cd ~/openshift/install
openshift-install wait-for bootstrap-complete --log-level=debug
openshift-install wait-for install-complete --log-level=debug

Важно. Параллельно с установкой кластера проверяйте процесс выдачи сертификатов.

Пример процесса выдачи сертификатов:

oc get csr
NAME        AGE     SIGNERNAME                                    REQUESTOR                                                                   CONDITION
csr-25z84   91m     kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-7jpl5   22m     kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Pending
csr-7r7z2   81m     kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-hwjbp   37m     kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Pending
csr-lxgq5   91m     kubernetes.io/kubelet-serving                 system:node:mst-1.prod.okd.lan                                              Approved,Issued
csr-r4jgl   81m     kubernetes.io/kubelet-serving                 system:node:mst-3.prod.okd.lan                                              Approved,Issued
csr-vvdbc   81m     kubernetes.io/kubelet-serving                 system:node:mst-2.prod.okd.lan                                              Approved,Issued
csr-wj5qn   7m18s   kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Pending
csr-wlxwh   81m     kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-xzmms   16m     kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Pending
csr-z99h9   31m     kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Pending
csr-zql4k   89s     kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Pending

Если обнаружены ожидающие запросы со статусом «Pending», то подтвердите их с помощью команды:

oc get csr -o go-template='{{range .items}}{{if not .status}}{{.metadata.name}}{{"\n"}}{{end}}{{end}}' | xargs oc adm certificate approve

  1. Убедитесь, что все узлы кластера находятся в состоянии Ready:

oc get nodes
NAME                 STATUS   ROLES    AGE   VERSION
mst-1.prod.okd.lan   Ready    master   45m   v1.19.2+f173eb4-1049
mst-2.prod.okd.lan   Ready    master   45m   v1.19.2+f173eb4-1049
mst-3.prod.okd.lan   Ready    master   45m   v1.19.2+f173eb4-1049
wrk-1.prod.okd.lan   Ready    worker   20m   v1.19.2+f173eb4-1049
wrk-2.prod.okd.lan   Ready    worker   26s   v1.19.2+f173eb4-1049

  1. Убедитесь, что все cluster-operators запущены и доступны:

[root@srv install]# oc get co
NAME                                                                   VERSION                         AVAILABLE   PROGRESSING   DEGRADED SINCE
authentication                                                         4.6.0-0.okd-2021-02-14-205305   True        False         False      10s
cloud-credential                                                       4.6.0-0.okd-2021-02-14-205305   True        False         False      11h
cluster-autoscaler                                                     4.6.0-0.okd-2021-02-14-205305   True        False         False      11h
config-operator                                                        4.6.0-0.okd-2021-02-14-205305   True        False         False      11h
console                                                                4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
csi-snapshot-controller                                                4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
dns                                                                    4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
etcd                                                                   4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
image-registry                                                         4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
ingress                                                                4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
insights                                                               4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
kube-apiserver                                                         4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
kube-controller-manager                                                4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
kube-scheduler                                                         4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
kube-storage-version-migrator                                          4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
machine-api                                                            4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
machine-approver                                                       4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
machine-config                                                         4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
marketplace                                                            4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
monitoring                                                             4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
network                                                                4.6.0-0.okd-2021-02-14-205305   True        False         False      11h
node-tuning                                                            4.6.0-0.okd-2021-02-14-205305   True        False         False      11h
openshift-apiserver                                                    4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
openshift-controller-manager                                           4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
openshift-samples                                                      4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
operator-lifecycle-manager                                             4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
operator-lifecycle-manager-catalog                                     4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
operator-lifecycle-manager-packageserver                               4.6.0-0.okd-2021-02-14-205305   True        False         False      10h
service-ca                                                             4.6.0-0.okd-2021-02-14-205305   True        False         False      11h
storage                                                                4.6.0-0.okd-2021-02-14-205305   True        False         False      11h

После выполнения действий будет развёрнут отказоустойчивый кластер на основе OKD/OCP. Перейдите к настройке кластера.

Пример вывода после установки кластера:

openshift-install wait-for install-complete --log-level=debug
DEBUG OpenShift Installer 4.6.0-0.okd-2021-02-14-205305
DEBUG Built from commit 20585a2903a61784703fde00e38af1208c007599
DEBUG Loading Install Config...                  
DEBUG   Loading SSH Key...                        
DEBUG   Loading Base Domain...                    
DEBUG     Loading Platform...                     
DEBUG   Loading Cluster Name...                   
DEBUG     Loading Base Domain...                  
DEBUG     Loading Platform...                     
DEBUG   Loading Pull Secret...                    
DEBUG   Loading Platform...                       
DEBUG Using Install Config loaded from state file 
INFO Waiting up to 40m0s for the cluster at https://api.prod.okd.lan:6443 to initialize...
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 0% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 7% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 8% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 63% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 64% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 66% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 86% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 86% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 86% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 86% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 86% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: downloading update
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 1% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 9% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 57% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 67% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 69% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 71% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 87% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 88% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 88% complete, waiting on authentication, console, ingress, kube-storage-version-migrator, monitoring
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 89% complete
DEBUG Still waiting for the cluster to initialize: Working towards 4.6.0-0.okd-2021-02-14-205305: 89% complete
DEBUG Cluster is initialized                      
INFO Waiting up to 10m0s for the openshift-console route to be created...
DEBUG Route found in openshift-console namespace: console
DEBUG Route found in openshift-console namespace: downloads
DEBUG OpenShift console route is created          
INFO Install complete!                           
INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/root/openshift/install/auth/kubeconfig'
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.prod.okd.lan
INFO Login to the console with user: "kubeadmin", and password: "PiXuD-zAUIt-3NTWU-JfIYo"
DEBUG Time elapsed per stage:                     
DEBUG Cluster Operators: 35m16s                   
INFO Time elapsed: 35m16s

См. также:

Подготовка и развёртывание отказоустойчивого кластера на основе OKD/OCP | Настройка кластера