Preparing Environment and Deployment of Auxiliary Nodes

To deploy a fault-tolerant cluster based OKD/OCP, prepare and environment and deploy auxiliary nodes:

1. Create a single VLAN OKD-prod in vCentre (WUI of virtualization environment) for cluster network.

2. Load and install the VMware template to the virtualization environment.

3. Reserve IP addresses for the srv and rt hosts if DHCP server is connected in the Office LAN network, or use static IP addressing.

4. Create the okd.lan zone on the DNS server in the Office LAN network and the prod domain contained in this zone. After this add DNS records:

NOTE. Domain names are specified for the example of environment preparation and deployment of auxiliary nodes and may differ.

5. Deploy auxiliary nodes:

• Router (rt).

• Service node (srv).

After executing the operations, the environment is prepared and auxiliary nodes are deployed for the cluster. Proceed to cluster deployment.

Router Deployment

To deploy router:

1. Create a virtual machine according to the system requirements and set up switching of virtual machine network interfaces according to the cluster components interaction scheme.

2. Load pfSense (architecture/AMD64) to iso disk and connect it to the virtual machine. For details about the use of pfSense see the documentation.

3. Start the virtual machine and install pfSense loaded from the iso disk. Use default values on installation.

4. Restart the virtual machine and disconnect the iso disk.

5. Open pFsense Console (Shell) and execute the following operations one by one:

• Create an allow rule to access the router from the Office LAN network using the command:

easyrule pass wan tcp any any 443

• Disconnect packet filter using the command:

pfctl –d

6. Connect to the WUI web interface at:

https://rt.prod.okd.lan

Execute the following operations in the WUI web interface one by one:

• Set host name and set up network interfaces.

• Disable blocking of IP addresses of private networks in the WAN interface using the command: Interfaces > WAN (vmx0) > Block RFC1918 Private Networks.

• Install Open-VM-Tools using the command: System > Package Manager > Available Package > Open-VM-Tools.

• Install updates using the command: System > Update > System Update.

After executing the operations the router is deployed.

Service Node Deployment

To deploy service node:

1. Create a virtual machine according to the system requirements and set up switching of virtual machine network interfaces according to the cluster components interaction scheme.

2. Load Fedora Server Standard (33/Server/x86_64) to iso disk and connect it to the virtual machine. For details about the use of Fedora Server Standard see the documentation.

3. Start the virtual machine and install the loaded Fedora Server Standard from the iso disk. Use default values on installation.

4. Restart the virtual machine and disconnect the iso disk.

5. Connect to the WUI web interface at:

https://srv.prod.okd.lan:9090

Execute the following operations in the WUI web interface one by one:

• Set host name and set up network interfaces.

• Install updates using the command: System > Update > System Update.

• Install the following applications:

• DNS server. Execute the commands:

sudo dnf -y install bind bind-utils
sudo cat <<EOF > /etc/named.conf
options {
        listen-on port 53 { any; };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
        secroots-file   "/var/named/data/named.secroots";
        recursing-file  "/var/named/data/named.recursing";
        allow-query     { any; };
        recursion yes;
        forwarders {
                10.9.0.5;
                10.9.0.6;
        };
        dnssec-enable yes;
        dnssec-validation no;
        managed-keys-directory "/var/named/dynamic";
        geoip-directory "/usr/share/GeoIP";
        pid-file "/run/named/named.pid";
        session-keyfile "/run/named/session.key";
        include "/etc/crypto-policies/back-ends/bind.config";
};
logging {
        channel default_debug {
                file "data/named.run";
                severity dynamic;
        };
};
zone "." IN {
        type hint;
        file "named.ca";
};
zone "prod.okd.lan" IN {
        type  master;
        file "zonefile.db";
};
zone "0.168.192.in-addr.arpa" IN {
        type master;
        file "reverse.db";
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";
EOF
sudo cat <<EOF > /var/named/zonefile.db
$TTL 86400
@       IN      SOA     ns1.prod.okd.lan.        root (
                        1614163446      ; serial
                        3H              ; refresh (3 hours)
                        30M             ; retry (30 minutes)
                        2W              ; expiry (2 weeks)
                        1W )            ; minimum (1 week)
        IN      NS      ns1.prod.okd.lan.
ns1     IN      A       192.168.0.2
api             IN      A       192.168.0.2
api-int         IN      A       192.168.0.2
*.apps          IN      A       192.168.0.2
srv     IN      A       192.168.0.2
bs      IN      A       192.168.0.10
mst-1           IN      A       192.168.0.11
mst-2           IN      A       192.168.0.12
mst-3           IN      A       192.168.0.13
wrk-1           IN      A       192.168.0.14
wrk-2           IN      A       192.168.0.15
etcd-0  IN      A       192.168.0.11
etcd-1  IN      A       192.168.0.12
etcd-2  IN      A       192.168.0.13
_etcd-server-ssl._tcp   IN      SRV     0 10 2380 etcd-0.prod.okd.lan.
_etcd-server-ssl._tcp   IN      SRV     0 10 2380 etcd-1.prod.okd.lan.
_etcd-server-ssl._tcp   IN      SRV     0 10 2380 etcd-2.prod.okd.lan.
EOF
sudo cat <<EOF > /var/named/reverse.db
$TTL 86400
@       IN      SOA     ns1.prod.okd.lan.        root (
                        1614163446      ; serial
                        3H              ; refresh (3 hours)
                        30M             ; retry (30 minutes)
                        2W              ; expiry (2 weeks)
                        1W )            ; minimum (1 week)
        IN      NS      ns1.prod.okd.lan.
2       IN      PTR     api.prod.okd.lan.
2       IN      PTR     api-int.prod.okd.lan.
2       IN      PTR     srv.prod.okd.lan.
10      IN      PTR     bs.prod.okd.lan.
11      IN      PTR     mst-1.prod.okd.lan.
12      IN      PTR     mst-2.prod.okd.lan.
13      IN      PTR     mst-3.prod.okd.lan.
14      IN      PTR     wrk-1.prod.okd.lan.
15      IN      PTR     wrk-2.prod.okd.lan.
EOF
sudo systemctl enable named; sudo systemctl start named
sudo firewall-cmd --permanent --add-port=53/udp
sudo firewall-cmd --reload

• Load balancer. Execute the commands:

sudo dnf -y install haproxy
sudo cat <<EOF  > /etc/haproxy/haproxy.cfg
global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     20000
    user        haproxy
    group       haproxy
    daemon
    stats socket /var/lib/haproxy/stats   
ssl-default-bind-ciphers PROFILE=SYSTEM   
ssl-default-server-ciphers PROFILE=SYSTEM
defaults
    mode                    tcp
    log                     global
    option                  httplog
    option                  dontlognull
    option               http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 20000
listen stats
    bind :9000
    mode http
    stats enable
    stats uri /
    monitor-uri /healthz
frontend openshift-api-server
    bind *:6443   
default_backend openshift-api-server
    option tcplog
backend openshift-api-server
    balance source
    server bs 192.168.0.10:6443 check
    server mst-1 192.168.0.11:6443 check
    server mst-2 192.168.0.12:6443 check
    server mst-3 192.168.0.13:6443 check
frontend machine-config-server
    bind *:22623   
default_backend machine-config-server
    option tcplog
backend machine-config-server
    balance source
    server bs 192.168.0.10:22623 check
    server mst-1 192.168.0.11:22623 check
    server mst-2 192.168.0.12:22623 check
    server mst-3 192.168.0.13:22623 check
frontend ingress-http
    bind *:80   
default_backend ingress-http
    option tcplog
backend ingress-http
    balance source
    server wrk-1-http 192.168.0.14:80 check
    server wrk-2-http 192.168.0.15:80 check
frontend ingress-https
    bind *:443   
default_backend ingress-https
    option tcplog
backend ingress-https
    balance source
    server wrk-1-https 192.168.0.14:443 check
    server wrk-2-https 192.168.0.15:443 check
EOF
sudo setsebool -P haproxy_connect_any 1 
sudo systemctl enable haproxy; sudo systemctl start haproxy 
sudo firewall-cmd --permanent --add-port=6443/tcp 
sudo firewall-cmd --permanent --add-port=22623/tcp 
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https 
sudo firewall-cmd --reload

• WEB server. Execute the commands:

sudo dnf install -y httpd
sudo sed -i 's/Listen 80/Listen 8080/' /etc/httpd/conf/httpd.conf
sudo setsebool -P httpd_read_user_content 1
sudo systemctl enable httpd; sudo systemctl start httpd
sudo firewall-cmd --permanent --add-port=8080/tcp
sudo firewall-cmd --reload

After executing the operations, the service node is deployed.

See also:

Preparation and Deployment of Fault-Tolerant Cluster Based on OKD/OCP | Deployment of Cluster Based on OKD/OCP

Copy page URL

Back to top

To report a found bug, please, select the text with the bug and click Report a Bug or CTRL+ENTER.

Help system 23.05 of 09/07/2024, © FORESIGHT