Redis is a key-value in-memory database (Redis.io). You can use Redis since version 10.0.1 in IBM Verify Access Manager as an alternative to the DSC (Distributed Session Cache) in WebSEAL and also in the Federation component as a replacement for the HVDB in some situations. Although it’s not possible yet to store everything in Redis , I think this will be definitely the case in the future!

There’s a couple of advantages to using Redis instead of the traditional DSC:

it no longer requires a cluster on ISAM - this is a huge advantage. In my opinion, you should not use ISAM clusters anymore nowadays.
it scales independently of ISAM

Redis clusters are not supported in ISAM, but Redis Sentinels are. So to achieve high availability, you must turn to a setup using at least 1 Redis replica with the supporting Redis Sentinels. Redis sentinel brings resilience to the solution, but it’s not a scaling solution.

Redis Sentinels work together with a master-replica setup.

Redis Sentinel is a separate service that runs alongside the Redis datastore and that monitors the health of the master. The Sentinels need to agree between themselves to promote a replica to master. To do that, they use a quorum mechanism. Because of this mechanism, you MUST have (at least) 3 sentinel servers, so you can set the required quorum to 2 (at least).

An important side note here is that the Redis client that is used should support Sentinel as well. The logic to switch to a new master, should be provided by the client. I don’t know what client ISAM uses, but it’s visible in the tracing what the logic is. See logging and tracing.

Redis

infrastructure

So you need (at least) 3 systems

1 master (+sentinel)
1 slave (+sentinel)
1 additional sentinel

Installation

In all cases, the redis package includes both the redis database and sentinel.

Red Hat Enterprise Linux 8

In RHEL 8, Redis 6 is supplied as a Yum module. Don’t install using ‘simple’ yum commands, because you’ll end up with Redis 5.

sudo yum module install "@redis:6/common"

Fedora

In Fedora, you can simply install Redis using yum or dnf.

sudo yum install redis

Base configuration

All redis servers should have the same configuration, except for the “replicaof” setting. These keys should be modified for a working setup:

bind <external ip address of your server> 127.0.0.1
port 6379
daemonize yes
protected-mode yes
supervised auto
replica-read-only yes

Specifically, the users that you’ve defined for replication, for sentinel and for your clients should be in ALL redis.conf files (both master and replica).

user default off nocommands allkeys >SOMERANDOMPASSWORD
user isam on +@all -CONFIG allkeys >ThePassWordForISAM
user sentinel-user on +client +subscribe +publish +ping +info +multi +slaveof +replicaof +config +client +exec >SENTINELRANDOMPASSWORD
user replica-user on +@all allkeys >REPLICARANDOMPASSWORD

NOTE If you are going to use the same Redis Sentinel setup for your Federation Runtime as well, you need to use a slightly different configuration for the default user:

user default on nopass -@all +hello

This has to do with the Lettuce Java Redis client that is used in the Liberty Runtime.

The masteruser/masterauth that are used for replication, should also be in ALL redis.conf files.

masteruser replica-user
masterauth REPLICARANDOMPASSWORD

This needs to be in the master’s redis.conf file as well, because when the failover happens the master becomes a replica and won’t be able to replicate if it doesn’t have the credentials.

Configuration for replica

The replica should be configured to connect to the host and port of the master.

replicaof <ip address of the master> 6397

This configuration parameter should not exist on the master, because that will break things.

Security

ACL

The ACL (the users we created in the redis.conf) does not magically make your Redis system secure, the passwords are stored in clear text in the configuration files and you can brute force Redis (if you can connect to it) - Redis is very fast and there is no rate limit protection.
A hacker could brute-force test 1000’s of passwords in seconds… So passwords should be very long (+20 characters).

That said, it’s still a good idea to disable the default unnamed user, and create separate users for the different tasks. So in this case, we’ll create a user for the ISAM systems to use, a user for Sentinel and a Replica user (for replication).

The isam user needs to have some specific rights as documented here: (https://www.ibm.com/docs/en/sva/10.0.3?topic=limiting-redis-commands)

user	role
isam	user for the isam client
sentinel-user	user for sentinels
replica-user	user for replicas

TLS

For a production setup, you really should configure TLS. I’ve not done that in this case (yet).

Firewall

You should have firewall rules in place so only systems that are allowed to connect to Redis, can connect to Redis.

Redis should definitely not be exposed to the internet!

Sentinel

The only configuration that is necessary on the sentinels, is connecting it to the master. Since we use protected mode, we also need to supply the authentication data for our sentinel-user.

#MUST use bind <external_ip> 127.0.0.1
bind <external ip address of your server> 127.0.0.1
port 26379
sentinel monitor mymaster <master ip address> 6379 2

sentinel auth-user mymaster sentinel-user
sentinel auth-pass mymaster SENTINELRANDOMPASSWORD
# Default is 30 seconds.
sentinel down-after-milliseconds mymaster 5000
# Default is 3 minutes - 180000
sentinel failover-timeout mymaster 9000

NOTE : the order of interfaces for bind is very important: the loopback interface should not be the first one!

You should check the sentinel log, to verify the sentinel can connect to the master, detects the other replica(s) and that it can detect failures.

It is possible to make a ‘cluster’ out of the setup, by using additonal components like haproxy : https://www.willandskill.se/en/setup-a-highly-available-redis-cluster-with-sentinel-and-haproxy.amp.html/.

I don’t think this is necessary in a scenario with IBM Access Manager - a relatively small Redis server should be able to store a very large number of concurrent ISAM/ISVA sessions (100000+) without issue.
Introducing components also makes the setup more complex and simpler is better.

Start Redis and Sentinels

To start everything, you can use the services that are created:

sudo systemctl start redis
sudo systemctl start redis-sentinel

You can also enable them to start on boot, if you like.

sudo systemctl enable redis
sudo systemctl enable redis-sentinel

ISAM/ISVA

Redis for WebSEAL

Redis can be used as a replacement for the DSC in WebSEAL. The configuration uses ISAM-specific terms as “redis collection” and “redis servers” .

The Redis client in WebSEAL uses some Redis commands to find what server is the master (specifically, the ROLE command) .

The Redis client in WebSEAL does not automatically “discover” the replica(‘s) in a Redis Sentinel setup, so you do have to define the Servers in the Collection, in ISAM.

Configuration in ISAM

The documentation is a bit sparse at the moment, although the actual implementation is pretty simple.

There’s a ‘wizard’ type of configuration, but this is not documented at the moment:

Create redis collections: go to Web/Redis configuration (https:///wga/redis_config/collections) Use the 'New' button to add a collection. You can just give it a name , and keep the defaults .

Image:Redis collection

Then, go to the ‘servers’ tab and add your Redis servers (not the Sentinels, only the servers).

This seems more like convenience than anything else , this is not strictly necessary (if you would choose to edit the webseald.conf by hand).

Edit your instance, go to the Redis tab

Image:Redis configuration

This creates the redis collection and server stanzas in the webseald.conf (you could just as well do that by hand)

Enable distributed sessions in the webseald.conf

Edit the session stanza:

[session] 
dsess-enabled = yes
dsess-server-type = redis

Restart the reverse proxies

NOTE : the mechanism in ISAM is the same or similar to what is used in IAG, and there’s better documentation on Redis with IAG : https://docs.verify.ibm.com/gateway/docs/tasks-sharing-sessions.

Test

If you now login to the reverse proxy, your session will be stored in Redis.

You can use a multitude Redis client to look at the session in the datastore, but the installation of redis comes with the redis-cli

Redis cli

redis-cli -h <master_ip> -p 6379

This connects to the redis server, so you can authenticate and run commands.

AUTH isam ThePassWordForISAM
KEYS *

You can now see the isam keys. Up to 3 keys can be created per user session (https://www.ibm.com/docs/en/sva/10.0.2?topic=overview-redis-keys)

Test session persistence on WebSEAL

If you don’t use DSC or Redis, you are logged out if the Reverse Proxy restarts.

So you can now restart the reverse proxy, and you will remain logged in.

Test fail over of Redis

To force a failover, you can use the DEBUG command. This will make the replica take over the role of the master, and demote the master to replica.

AUTH isam ThePassWordForISAM
DEBUG sleep 30

During this action, you should remain logged in to ISAM.

After the failure occured, the master should now be a replica. You can check that using the info command. If you still have the redis-cli open, you’ll have to login again.

AUTH isam ThePassWordForISAM
info replication

Logging & Tracing

Logging

First the bad news : there’s hardly any logging in the webseal log for Redis. If things don’t work, you’re in the dark - it’s not possible to determine from the webseal log that there even is a problem. Users will experience an HTTP 503 code and will not be able to login nor continue existing sessions.

Tracing

You can enable the pdweb.redis trace setting to see what actually happens. Go to Web/Reverse Proxy. Select the reverse proxy you enabled Redis for and select ‘Troubleshooting/Tracing’

Find pdweb.redis and enable it.

Image:Redis tracing

Trace file example

This example shows what happens and how the Redis client in ISVA determines that it’s connected to the “master” or the “replica” (by issuing the ROLE command)

2022-04-25-13:08:18.003+00:00I----- thread(6) trace.pdweb.redis.mgr:5 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:388: 0x7fe65002dda0: Connected to the redis server: redis1.domain.com:6379
2022-04-25-13:08:18.003+00:00I----- thread(6) trace.pdweb.redis.mgr:7 /build/isam/src/i4w/pdwebrte/redis/AMWRedisPool.cpp:366: 0xf80c30: Created a new connection to redis1.domain.com:6379.
2022-04-25-13:08:18.003+00:00I----- thread(6) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:586: 0x7fe65002dda0: Executing the redis command: ROLE.
2022-04-25-13:08:18.004+00:00I----- thread(6) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:648: 0x7fe65002dda0: The command succeeded.
2022-04-25-13:08:18.004+00:00I----- thread(6) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:443: 0x7fe65002dda0: Reply: type[-1]: 5.
2022-04-25-13:08:18.004+00:00I----- thread(6) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:443: 0x7fe65002dda0: Reply: type[0]: 2.
2022-04-25-13:08:18.004+00:00I----- thread(6) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:537: 0x7fe65002dda0: Reply: string[0]: slave.
2022-04-25-13:08:18.004+00:00I----- thread(6) trace.pdweb.redis.mgr:8 /build/isam/src/i4w/pdwebrte/redis/AMWRedisCollection.cpp:500: 0x12522b0: The server is not a master!
2022-04-25-13:08:19.705+00:00I----- thread(5) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:796: 0x7fe650001580: Failed to retrieve the next reply: Server closed the connection
2022-04-25-13:08:19.705+00:00I----- thread(5) trace.pdweb.redis.mgr:7 /build/isam/src/i4w/pdwebrte/redis/AMWRedisSubscription.cpp:224: 0xf863d0: The connection for the __redis__:invalidate channel to the collection1 collection has finished.
2022-04-25-13:08:20.000+00:00I----- thread(5) trace.pdweb.redis.mgr:8 /build/isam/src/i4w/pdwebrte/redis/AMWRedisCollection.cpp:310: 0x12522b0: Checking the health of the collection: collection1
2022-04-25-13:08:20.000+00:00I----- thread(5) trace.pdweb.redis.mgr:8 /build/isam/src/i4w/pdwebrte/redis/AMWRedisCollection.cpp:465: 0x12522b0: Checking the health of the redis1.domain.com:6379 server in the collection1 collection.
2022-04-25-13:08:20.000+00:00I----- thread(5) trace.pdweb.redis.mgr:5 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:243: 0x7fe650001580: Creating a connection to the redis server: redis1.domain.com:6379
2022-04-25-13:08:20.004+00:00I----- thread(5) trace.pdweb.redis.mgr:5 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:344: 0x7fe650001580: Attempting to authenticate to the redis server.
2022-04-25-13:08:20.004+00:00I----- thread(5) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:688: 0x7fe650001580: Executing the redis command: AUTH xxx
2022-04-25-13:08:20.007+00:00I----- thread(5) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:749: 0x7fe650001580: The command succeeded.
2022-04-25-13:08:20.007+00:00I----- thread(5) trace.pdweb.redis.mgr:5 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:367: 0x7fe650001580: Successfully authenticated to the redis server.
2022-04-25-13:08:20.007+00:00I----- thread(5) trace.pdweb.redis.mgr:5 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:374: 0x7fe650001580: Attempting to set the client name: isva-3861eec8-c492-11ec-b2f7-0050569e684a.
2022-04-25-13:08:20.007+00:00I----- thread(5) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:586: 0x7fe650001580: Executing the redis command: CLIENT SETNAME isva-3861eec8-c492-11ec-b2f7-0050569e684a.
2022-04-25-13:08:20.009+00:00I----- thread(5) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:648: 0x7fe650001580: The command succeeded.
2022-04-25-13:08:20.009+00:00I----- thread(5) trace.pdweb.redis.mgr:5 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:388: 0x7fe650001580: Connected to the redis server: redis1.domain.com:6379
2022-04-25-13:08:20.009+00:00I----- thread(5) trace.pdweb.redis.mgr:7 /build/isam/src/i4w/pdwebrte/redis/AMWRedisPool.cpp:366: 0xf80c30: Created a new connection to redis1.domain.com:6379.
2022-04-25-13:08:20.009+00:00I----- thread(5) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:586: 0x7fe650001580: Executing the redis command: ROLE.
2022-04-25-13:08:20.010+00:00I----- thread(5) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:648: 0x7fe650001580: The command succeeded.
2022-04-25-13:08:20.010+00:00I----- thread(5) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:443: 0x7fe650001580: Reply: type[-1]: 5.
2022-04-25-13:08:20.010+00:00I----- thread(5) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:443: 0x7fe650001580: Reply: type[0]: 2.
2022-04-25-13:08:20.010+00:00I----- thread(5) trace.pdweb.redis.mgr:6 /build/isam/src/i4w/pdwebrte/redis/AMWRedisConnection.cpp:537: 0x7fe650001580: Reply: string[0]: master.