In this post I will explain how to set up a WSO2 Dashboard Server 2.0.0 (referred as DS hereafter) cluster in a distributed manner with two DS instances fronted with a Nginx Plus Load Balancer. The following diagram depicts the overall distribution of the cluster components.
Both DS instances of the cluster will act as manager nodes and the load balancer will distribute the load between these two nodes in a round robin fashion. Following are the pre-requisites to start off with this tutorial and the versions I used for this setup.
- Nginx Plus installed (1.7.11.- nginx-plus-r6-p1)
- Oracle setup (Oracle 12c)
- SVN server to use as the deployment synchronizer (v1.8)
- Dashboard Server 2.0.0 downloaded in both nodes.
Configuring the Load Balancer
I assume you have installed Nginx Plus by now. Follow the below steps to configure Nginx
1. Navigate to the following location
/etc/nginx/conf.d
2. Create a file named ds.conf (you can also use the default.conf file which is available there by default)
3. Add the following content and save the file
upstream httpdsportal {
server 192.168.48.76:9763;
server 192.168.48.77:9763;
}
upstream httpsdsportal {
server 192.168.48.76:9443;
server 192.168.48.77:9443;
sticky learn create=$upstream_cookie_jsessionid
lookup=$cookie_jsessionid
zone=client_sessions_ds:1m;
}
server {
listen 80;
server_name ds.wso2.com;
location / {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_read_timeout 5m;
proxy_send_timeout 5m;
proxy_pass http://httpdsportal/;
proxy_redirect http://httpdsportal/ http://ds.wso2.com/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
server {
listen 443;
server_name ds.wso2.com;
ssl on;
ssl_certificate /etc/nginx/ssl/ds/ds.crt;
ssl_certificate_key /etc/nginx/ssl/ds/ds.key;
location / {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_read_timeout 5m;
proxy_send_timeout 5m;
proxy_pass https://httpsdsportal/;
proxy_redirect https://httpsdsportal/ https://ds.wso2.com/;
}
4. I have created a ssl certificate (ds.crt) and a key (ds.key) and pointed to the location as you can see in the configuration above. Follow the steps in the link below to create the SSL certificate.
https://docs.wso2.com/display/CLUSTER44x/Configuring+NGINX
5. Finally restart the Nginx server.
sudo service nginx restart
If you have done the configurations correctly, the server will restart successfully.
Configuring the DS instances
First of all copy the cert file you created above (ds.crt) from nginx to following path in both DS instances.
DS_HOME/repository/resources/security
Then you need to import the certificate to the keystore.
keytool -import -alias <alias> -file ds.crt -keystore client-truststore.jks -storepass wso2carbon
Now, you are done importing your trusted certificate to the client trust store where all WSO2 products refer for trusted certificates. Follow the below steps to configure other files within the DS product.
axis2.xml configurations
Open DS_HOME/repository/conf/axis2/axis2.xml file and follow the steps below.
1. Enable clustering.
<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent"
enable="true">
2. Set the membership scheme to wka to enable the well-known address registration method
<parameter name="membershipScheme">wka</parameter>
3. Specify the name of the cluster this node will join
<parameter name="domain">wso2.ds.domain</parameter>
4. Specify the hosts / IP addresses
Node 1:
<parameter name="localMemberHost">192.168.48.76</parameter>
Node 2:
<parameter name="localMemberHost">192.168.48.77</parameter>
5. Specify the port used to communicate cluster messages
Node 1:
<parameter name="localMemberPort">4100</parameter>
Node 2:
<parameter name="localMemberPort">4200</parameter>
6. Specify the well known members
Node 1:
<members>
<member>
<hostName>192.168.48.77</hostName>
<port>4200</port>
</member>
</members>
Node 2:
<members>
<member>
<hostName>192.168.48.76</hostName>
<port>4100</port>
</member>
</members>
carbon.xml Configurations
Open DS_HOME/repository/conf/carbon.xml file and follow the steps below.
1. Configure the HostName and MgtHostName
<HostName>ds.wso2.com</HostName>
<MgtHostName>ds.wso2.com</MgtHostName>
2. Enable SVN-based deployment synchronization in both nodes. (Both nodes have read/write permission)
<DeploymentSynchronizer>
<Enabled>true</Enabled>
<AutoCommit>true</AutoCommit>
<AutoCheckout>true</AutoCheckout>
<RepositoryType>svn</RepositoryType>
<SvnUrl>URL</SvnUrl>
<SvnUser>username</SvnUser>
<SvnPassword>password</SvnPassword>
<SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer>
3. Download
http://product-dist.wso2.com/tools/svnkit-all-1.8.7.wso2v1.jar and install it by copying it into the<DS_HOME>/repository/components/dropins folder.
4. Download
http://maven.wso2.org/nexus/content/groups/wso2-public/com/trilead/trilead-ssh2/1.0.0-build215/trilead-ssh2-1.0.0-build215.jar and copy it to the <DS_HOME>/repository/components/lib folder.
catalina-server.xml Configurations
Open DS_HOME/repository/conf/tomcat/catalina-server.xml file and follow the steps below.
1. Configure the proxy ports
<Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
port="9763"
proxyPort="80"
...
/>
<Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
port="9443"
proxyPort="443"
...
/>
Configuring the Databases and Registry Mounting
Create two databases (this tutorial uses Oracle 12c) to store Registry and User information. Then, point the servers to the newly created databases as follows.
master-datasources.xml Configurations
Open DS_HOME/repository/conf/datasources/master-datasources.xml file and follow the steps below.
1. Add the data sources for Registry and User databases
<datasource>
<name>WSO2_DS_USER_DB</name>
<jndiConfig>
<name>jdbc/DS_USER_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
<url>url</url>
<maxActive>100</maxActive>
<maxWait>60000</maxWait>
<minIdle>5</minIdle>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1 FROM DUAL</validationQuery>
<validationInterval>30000</validationInterval>
<username>username</username>
<password>password</password>
<defaultAutoCommit>false</defaultAutoCommit>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2_DS_REGISTRY_DB</name>
<jndiConfig>
<name>jdbc/DS_REGISTRY_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
<url>url</url>
<maxActive>100</maxActive>
<maxWait>60000</maxWait>
<minIdle>5</minIdle>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1 FROM DUAL</validationQuery>
<validationInterval>30000</validationInterval>
<username>username</username>
<password>password</password>
<defaultAutoCommit>false</defaultAutoCommit>
</configuration>
</definition>
</datasource>
2. Copy the ojdbc7.jar driver for the oracle 12c database to <DS_HOME>/repository/components/lib directory.
registry.xml Configurations
Open DS_HOME/repository/conf/registry.xml file and follow the steps below.
1. Add/Update the following configurations
<dbConfig name="sharedregistry">
<dataSource>jdbc/DS_REGISTRY_DB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9443/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<cacheId>unique cache id</cacheId>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/config</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>
user-mgt.xml Configurations
Open DS_HOME/repository/conf/user-mgt.xml file and follow the steps below.
1. Point the user store to the newly created database.
<Configuration>
<AddAdmin>true</AddAdmin>
<AdminRole>admin</AdminRole>
<AdminUser>
<UserName>admin</UserName>
<Password>admin</Password>
</AdminUser>
<EveryOneRoleName>everyone</EveryOneRoleName> <!-- By default users in this role sees the registry root -->
<Property name="isCascadeDeleteEnabled">true</Property>
<Property name="dataSource">jdbc/DS_USER_DB</Property>
</Configuration>
designer.json configuration
Open DS_HOME/repository/deployment/server/jaggeryapps/portal/configs/designer.json file.
1. Configure the hostname of the cluster, https/https protocol and the port (optional in our setup)
"host": {
"hostname": "ds.wso2.com",
"port": "443",
"protocol": "https"
}
Without this configuration the gadgets added to the dashboard will not render in the dashboard. Also, you can access the dashboard using the IP address instead of the host name without any issue with this configuration.
hosts configuration
Open /etc/hosts file and do the following.
1. Map the host name of the cluster to the IP address of the Nginx load balancer as follows.
192.168.48.75 ds.wso2.com
This should be done in all the following nodes.
- Nginx plus node
- Both DS nodes
- Any server that is browsing the DS portal or management console
Starting the server
Now you have successfully configured all the configurations in order to start the DS servers.
sh wso2server.sh -Dsetup
2. To start the pack normally.
sh wso2server.sh or sh wso2server.sh start
Once started you can access the portal and the management console of DS as follows
- https://ds.wso2.com/portal
- https://ds.wso2.com/carbon
That's it. Now you can create dashboards with lots of new features using the WSO2 Dashboard Server. Please drop a comment if you have any queries :)
