Creating BI Servers Cluster

Several BI servers of Foresight Analytics Platform can be used to deploy a fault-tolerant cluster. One will require a balancer, state server and several BI servers configured for working with one state server. Links between all cluster elements can be shown as the following schema:

A balancer can be set up both before web servers and BI servers. As a rule, it is enough to set up a balancer before BI server and set up a web application for the balancer address. Sometimes a web application can be a bottleneck, in this case it is necessary to increase the number of web servers and add the balancer before them. Therefore, users will open the address of the balancer, which will direct queries to one of the web services.

NOTE. Before creating a cluster see general recommendations. When working with Linux OS see the nginx build process with the sticky module.

Consider deploying a cluster based on several BI servers of Foresight Analytics Platform, balancer, and the Memcached state server. To create a cluster, determine the following settings:

After installing BI servers, change the settings.xml configuration file for each of them:

In the Statable parameter set the Yes value.
In the StateServer section specify state server parameters.

NOTE. Working with the state server is available if monikers are generated by the BI server via GUID, that is why make sure that the KeyIncr parameter is set to 0 in the configuration file.

Example:

<...>
    <Key Name="PP">
        <BIS>
            <Key Name="System">
                <...>
                <Session ... Statable="Yes" ... />
                <StateServer Host="StateServer" Port="11211" Type="MemCached" />
                <...>
            </Key>
        </BIS>
    </Key>
</...>

Install HAProxy from repository:

sudo apt install haproxy

To use HAProxy balancer with BI servers of Foresight Analytics Platform, edit the /etc/haproxy/haproxy.cfg configuration file after the balancer is installed. Below is the example of the file with balancing set up for four B servers, the Round-robin algorithm is used, and balancer statistics is displayed at the 8404 port:

global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
user haproxy
group haproxy
maxconn 120000 # Maximum number of connections recommended by vendor.
daemon
# Default SSL material locations.
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3
defaults
log global
mode http
option httplog # Extended HTTP log.
option dontlognull # Disable logging of the connections, to which data was not sent.
timeout connect 60s
timeout client 500s
timeout server 500s
timeout http-keep-alive 100s
# Default error files. HAProxy sends them according to configured rules, and BI servers did not responded.
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
frontend ft_bi
bind 0.0.0.0:555 #Port for balancer work.
bind-process all #It attaches port listening to specific process numbers. The all value by default. It can be omitted.
default_backend bk_bi #Group of BI servers.
http-response set-header Keep-Alive timeout=100 # Send in responses the keep-alive timeout value on balancer side (in seconds).
backend bk_bi #Name of group of BI servers.
option forwardfor
balance roundrobin #Balancing algorithm.
cookie IDBI insert indirect nocache #Add cookie to provide sticky session mode.
default-server check inter 2s rise 2 fall 3 #Enable performance check for all BI servers. Check each two seconds. Treat BI server as crashed after three failed checks and restored after two successful checks. Crashed BI servers are excluded from balancing.
default-server maxconn 64 # Enable for servers maximum number of connections, 64 by default (Default number of workers in Apache configuration file). If all BI servers are used, connections are queued. If queue time exceeds timeout connect, the balancer returns the 503 error.
server bi1 127.0.0.1:81 cookie sb1 #BI server name and address. One should write own name and address.
server bi2 127.0.0.1:82 cookie sb2
server bi3 127.0.0.1:83 cookie sb3
server bi4 127.0.0.1:84 cookie sb4
frontend stats #Configure balancer statistics page.
bind *:8404
stats enable
stats uri /stats
stats refresh 10s

If HAProxy version from repository is not suitable for some reason, one can build the required version from source files. Links to various HAProxy versions can be found at the official website. One can also use the official repository at GitHub.

To use nginx balancer together with BI servers of Foresight Analytics Platform after installation, edit the nginx.conf configuration file (located in the ...\nginx\conf\ folder in Windows OS or the /etc/nginx/ folder in Linux OS). The file should have the following contents:

worker_processes  1;
worker_rlimit_nofile 20240;
events {
    worker_connections  20240;
}
http {
    log_format upstream   '$upstream_addr - $upstream_status | $request - [ $upstream_response_time ]';
    upstream backend {
        sticky;
        server <BIServer1>;
        server <BIServer2>;
    }
    server {
        listen   555;
        server_name  <NginxServerName>;
        access_log   logs/nginx_upstream_access.log  upstream;
        location / {
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $host:555;
            proxy_pass  http://backend;
        }
    }
}

Set up balancer parameters:

In the upstream directive set the list of servers, to which queries are forwarded. Specify the name, which will be used in the "location" directive, after the "upstream" keyword:

Set the "sticky" directive to specify that all queries of one client are linked to and sent to one BI server selected according to configured balancing method. If the assigned server cannot handle a query, a new server is selected as if the client is not linked to the server. The information about the server, to which queries must be sent, is transferred via random data sets (cookies), which are generated for each client. To make this directive work, in nginx set up the nginx-sticky additional module (it is not included in the build by default).
In the server directives set names or IP addresses of BI servers, to which queries will be forwarded.

In the "server" directive set parameters of the virtual server, which the web application will address:

In the "listen" directive set the port, via which the work will be executed.
In the "server_name" directive set name of the server, on which nginx is running.
In the "location" directive set query forwarding rules. For details about IP address logging depending on balancer location, see the Setting Up IP Address Logging in Linux OS section.
In the "proxy_pass" directive set the section that describes the servers, to which forwarding should be executed.

For example, the following values are specified in the configuration file:

upstream backend {
    sticky;
    server BIServer1;
    server BIServer2;
}
server {
    listen   555;
    server_name  BIServer;
    access_log   logs/nginx_upstream_access.log  upstream;
    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host:555;
        proxy_pass  http://backend;
    }
}

In this case all queries received at http://BIServer:555/axis2/services/PP.SOM.Som will be forwarded to BI servers at http://BIServer1/axis2/services/PP.SOM.Som and http://BIServer2/axis2/services/PP.SOM.Som/.

For details about parameters specified in the nginx.conf configuration file see the nginx documentation.

To make the web application work via the nginx balancer, in the PP.xml file specify the path and port to the balancer instead of the BI server:

<?xml version="1.0" encoding="utf-8" ?>
<pp>
    <service url="http://BIServer:555/axis2/services/PP.SOM.Som" timeout="120000"/>
    <metabase id="Warehouse" ping="120000" recentObjects="false" />
    <cultures>
        <culture cultureName="ru" />
        <culture cultureName="en" />
    </cultures>
    <...>
</pp>

Therefore, a query from the client goes through the web server to the balancer. The balancer selects the BI server, and all queries from this client will be sent to the selected BI server. The BI server after query handling and referring to the DBMS will save parameters of all objects at the state server. If on executing a query the BI server failure occurred, the balancer will resend the query to another BI server, which will restore all necessary objects from the state server, execute the query and create the report for the user.

If the user plans to work with web forms that use the WebSocket protocol, it is required to use the HAProxy balancer. For details about balancer setup see the Setting Up HAProxy Balancer for Working with Web Forms article.

Creating BI Servers Cluster

Cluster Standard Operating Mode

Failure of Sending Query to Cluster