HA Redis Configuration

HA Redis Configuration
Aspera / IBM / Tips

HA Redis Configuration

Introduction:


HSTS 4.1.1 included a re-architected of HA Redis this configuration is for 4.1.1 or later. The configuration steps below apply to new installs as well as the previously configured cluster being upgraded from a version prior to 4.1.1. The cluster consists of an odd number of 3 or more nodes. Each node needs its own separate perpetual license or entitlement. Also, ensure your node’s local IP Addresses are static or elastic.

Configuration:

Stop services on all nodes

Apply aspera.conf changes to all nodes setting the nodes to ha_redis, confirming the IP Addresses of the nodes and the port to be used (examples replace nodeX-ip with nodes local Address port can be different but suggested 41415):

# asconfigurator -x “set_server_data;db_type,ha_redis”

# asconfigurator -x ‘set_server_data;db_host,”node1-ip,node2-ip,node3-ip”‘

# asconfigurator -x ‘set_server_data;db_port,”41415,41415,41415″‘

Configure the Primary nodes /opt/aspera/etc/redis/aspera_31415.conf example (replace node1-IP with the Local IP of node1) :

port 31415

dbfilename redis.31415.rdb

dir /opt/aspera/var

pidfile /opt/aspera/var/run/redis.31415.pid

save 60 10000

daemonize no

syslog-enabled yes

syslog-facility local2

bind node1-ip

Then configure all other Secondary nodes /opt/aspera/etc/redis/aspera_31415.conf example (replace Secondary-IP with the Local IP of the secondary node and node1-ip with the Primary Nodes IP leave port 31415 as is):

port 31415

dbfilename redis.31415.rdb

dir /opt/aspera/var

pidfile /opt/aspera/var/run/redis.31415.pid

save 60 10000

daemonize no

syslog-enabled yes

syslog-facility local2

bind Secondary-IP

slaveof node1-ip 31415

Configure the Primary nodes Redis Sentinel in file /opt/aspera/etc/redis/sentinel_41415.conf example (replace node1-IP with the Local IP of node1) :
pidfile /opt/aspera/var/run/redis_sentinel_41415.pid
daemonize no
dir /tmp
port 41415
bind node1-IP
sentinel monitor mymaster node1-IP 31415 2
sentinel down-after-milliseconds mymaster 300
sentinel failover-timeout mymaster 1000
syslog-ident sentinel
syslog-enabled yes
syslog-facility local2

Configure all other Secondary nodes /opt/aspera/etc/redis/aspera_31415.conf example (replace s with the Local IP of the secondary nodes and node1-IP with the Local IP of the PrimaryNode):
pidfile /opt/aspera/var/run/redis_sentinel_41415.pid
daemonize no
dir /tmp
port 41415
bind Secondary-IP
sentinel monitor mymaster node1-IP 31415 2
sentinel down-after-milliseconds mymaster 300
sentinel failover-timeout mymaster 1000
syslog-ident sentinel
syslog-enabled yes
syslog-facility local2

Set the file perms and confirm full dir perms with these command:
chown -R asperadaemon:aspadmins /opt/aspera/etc
chown -R asperadaemon:aspadmins /opt/aspera/var

On all of the nodes (the primary and secondary nodes), enable the asperaredissentinel service.Run the following commands:
# cp /opt/aspera/etc/redis/asperaredissentinel.service /lib/systemd/system/ 

# systemctl enable asperaredissentinel 

Confirm the firewall rules allow for communication between nodes on TCP ports 31415 and 41415. 

Start up services a node at a time starting with the Primary node confirming (I suggest running status and confirming/addressing any errors before moving to the next service or node):
# systemctl start asperaredisd
# systemctl start asperaredissentinel
# systemctl start asperanoded
# systemctl start asperarund

Test HA Redis Failover. If you run the sleep command below and a new node doesn’t become the master. HA Redis Failover is not working and there is likely a config file mistake, permission issue or firewall blocking ports:

  • Confirm what node is master:
    /opt/aspera/bin/asredis -p 41415 -h localIPofAnyNode SENTINEL get-master-addr-by-name mymaster
  • Put the master to sleep causing failure (replace MasterIP with the output from the prior command)
    • /opt/aspera/bin/asredis -p 31415 -h MasterIP DEBUG sleep 30
  • Run this again to confirm the mymaster IP changed /opt/aspera/bin/asredis -p 41415 -h localIPofAnyNode SENTINEL get-master-addr-by-name mymaster

Now set the cluster ID Master:

  • This needs to be the same on all nodes, level them to one of the nodes files:
    /opt/aspera/etc/conf.d/cluster_id.conf

Only on the current MasterNode run the following to set the redismaster and make asperanoded start as master otherwise they are all slaves for asperanoded processes
yum install -y jq

Replacing node_aspera:password with your node username and password and node1-IP with the local IP of the Primary Node:
node_id=`curl -ks -unoda_aspera:password https://localhost:9092/info | jq -r ‘.node_id’` && echo “set skv:clmaster:str $node_id” | /opt/aspera/bin/asredis -h node1-IP -p 31415

systemctl restart asperanoded

Confirm asperanoded starts with a status of “New node state Master”
You can confirm if with this command:
grep New node state /var/log/aspera.log ( or messages if you have not turned on aspera.log filter)

Your HA Redis is successfully configured.

To learn more about PacGenesis, follow @PacGenesis on Facebook, Twitter, and LinkedIn, or visit us at pacgenesis.com.

512-766-8715

Download our latest Technology Brief

Learn more about how IBM Aspera can help you work at the speed of your ideas.

Schedule Dedicated Time With Our Team

Take some time to connect with our team and learn more about the session.