Preparing Environment for Foresight Mobile Platform

To prepare environment for Foresight Mobile Platform:

  1. Prepare cluster nodes.

  2. Install Heartbeat package.

  3. Prepare Ceph distributed file cluster.

  4. Prepare Kubernetes cluster.

  5. Install Rancher to manage k8s cluster.

NOTE. Main cluster nodes will be further designated as kn0, kn1, kn2, work nodes will be designated as kn3, kn4, kn5.

Preparing Cluster Nodes

To prepare cluster nodes:

  1. Create the fmpadmin user at all nodes who will be used to set and control the future cluster:

sudo useradd -m -d /home/fmpadmin -s /bin/bash fmpadmin
# Set password for fmpadmin user
sudo passwd fmpadmin

Add the fmpadmin user to sudo users to avoid repeated entering passworff when using the sudo command:

echo "fmpadmin ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/fmpadmin
chmod 0440 /etc/sudoers.d/fmpadmin

  1. Disable SWAP partition on all cluster nodes because kubelet does not support work with SWAP:

sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
swapoff –all

IMPORTANT. If all cluster nodes are virtual machines operated with VMware, install vmware-tools package on each node:
sudo apt-get install -y open-vm-tools

  1. Install basic packages and environment of Docker Community Edition:

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update

To view the list of Docker Community Edition versions available for installation, execute the command:

sudo apt-cache madison docker-ce

Example of command execution result:

###On each of your machines, install Docker. Version 18.09.9 is recommended, but 1.11, 1.12, 1.13, 17.03 and 19.03.5 are known to work as well.
### Latest validated Docker version: 18.09.x
## Install docker ce.

sudo apt-get update && sudo apt-get install docker-ce=5:18.09.9~3-0~ubuntu-xenial

  1. Add the fmpadmin user to the docker group:

sudo usermod -aG docker fmpadmin

  1. Set up the file /etc/hosts identically at all nodes:

127.0.0.1      localhost
<host-ip-address>    kn0.our.cluster kn0
<host-ip-address>    kn1.our.cluster kn1
<host-ip-address>    kn2.our.cluster kn2
<host-ip-address>    kn3.our.cluster kn3
<host-ip-address>    kn4.our.cluster kn4
<host-ip-address>    kn5.our.cluster kn5

# The following lines are desirable for IPv6 capable hosts

#::1     localhost ip6-localhost ip6-loopback
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters

  1. Generate SSH key for the fmpadmin user at the kn0 node:

    1. Execute the command as the fmpadmin user:

ssh-keygen

The dialog is displayed in the console, for example:

Enter file in which to save the key (/home/user/.ssh/id_rsa):

Press the ENTER key. Then it is prompted to enter a pass phrase for additional protection of SSH connection:

Enter passphrase (empty for no passphrase):

Skip this step and the next step. To do this, press the ENTER key. As a result, a SSH key is created.

    1. Create a configuration file for SSH:

vim ~/.ssh/config

Configuration file contents:

Host kn0
       Hostname kn0
       User fmpadmin
Host kn1
       Hostname kn1
       User fmpadmin
Host kn2
       Hostname kn2
       User fmpadmin
Host kn3
       Hostname kn3
       User fmpadmin
Host kn4
       Hostname kn4
       User fmpadmin

Host kn5
       Hostname kn5
       User fmpadmin

Save changes and exit the editor.

    1. Add read, write and execute permissions for the config file:

chmod 644 ~/.ssh/config

  1. Collect public keys of all nodes (it is executed on the kn0 node under the fmpadmin user):

$ ssh-keyscan kn0 kn1 kn2 kn3 kn4 kn5 >> ~/.ssh/known_hosts

  1. Add the created SSH key to all nodes:

ssh-copy-id kn1
ssh-copy-id kn2
ssh-copy-id kn3
ssh-copy-id kn4
ssh-copy-id kn5

NOTE. When password is requested, enter the password of the fmpadmin user.

  1. Set up NTP server to synchronize time between the nodes:

sudo apt-get install -y ntp ntpdate
cp /etc/ntp.conf /etc/ntp.conf.orig

NOTE. The setup should be executed for the second interface that is used only for inter-node traffic.

At the kn0 node:

cat << EOF > /etc/ntp.conf \
server 127.127.1.0 prefer \
fudge 127.127.1.0 stratum 10 \
interface ignore wildcard \
interface listen <ntp server lan ip address> \
EOF
systemctl restart ntp.service

At other nodes:

server <ntp server lan ip address on kn0 node> iburst
restrict default
interface ignore wildcard
interface listen <ntp client lan ip address>
sudo systemctl restart ntp.service

After executing the operations the clusters are prepared, next install Heartbeat package.

See also:

Preparation and Deployment of Fault-Tolerant Cluster | Preparing to Start Foresight Mobile Platform