Tips & tricks for installing and running IBM products

The only correct way to setup the ISAM RTE with basic users for the UserLookupHelper

Tom Bosmans  2 August 2019 09:03:41
I've been struggling a bit to get the UserLookupHelper work correctly in a custom authenticatiion mechanism I am building for Username / Password authentication.  I am using the ISAM "all-in-one" deployment pattern , that is becoming quite popular these days.  This basically consists of identical ISAM appliances that have all functions (reverse proxy, AAC, federation) and operate independently.  The environment provides high availability by adding identical appliances (typically using automation).  The identital appliances do share some resources, notably the High Volume database and the federated ldap repositories.
This pattern also relies on using Basic users in Federated repositories.

The problem I came across is that in this setup, the UserLookupHelper is very difficult to get to working correctly using the ISAM RTE.
Recentlly a technote was published (
https://www-01.ibm.com/support/docview.wss?uid=ibm10884160 ) , but that only describes what is causing the problem, not the detailed steps to resolve or avoid it.

I tried various workarounds, and a workaround that is suggested a lot is to use an LDAP connection to (1 of the) federated repositories.   While that may work for you, it is NOT what I want - I want to use ALL of the configured federated directories .

In the end , I found that a simple recommendation in setting up ISAM can avoid all these problems, so I'll start with that

Recommendation


In the "all-in-one" deployment pattern, with Basic users in remote ldap repositories, there is only 1 correct ISAM Runtime server setup : Local Policy Server/Remote LDAP Server (lmi / ssl) .

When initially configuring the Runtime environment, select a local Policy server and a remote LDAP server pointing to your embedded ldap (instead of local Policy Server/local LDAP server).
Configure the connection to your local embedded LDAP using:
- hostname: your appliance's LMI hostname
- port: 636
- enable ssl

Select a keystore that contains the CA certificates of ALL the federated repositories you want to connect to and also the CA for your embedded ldap.

The embedded LDAP does not listen on port 636 on the localhost/loopback/127.0.0.1 interface !


You need to make 2 small modification afterwards to ldap.conf

[ldap]

host =

port = 389
-> port = 636

ssl-port = 636


And under [bind-credentials], you must add the bind-dn and bind-pwd for a user that can read your embedded ldap (eg. the root user), like this :

[bind-credentials]

bind-dn = cn=root,secAuthority=Default

bind-pwd = your-root-password-i-hope-it-is-not-passw0rd



See the technote for the explanation.

This setup will enable you to use :
- UserName / Password AAC Authentication mechanism using the ISAM RTE
- the UserLookupHelper connecting to ISAM RTE (and all the configured federated repositories with basic users)
- the SCIM prerequisites

Errors & what does not work


UserLookupHelper does not work


So your userlookuphelper does not work in your setup with Basic Users - it cannot contact the ISAM RTE or it cannot contact the federated repositories you defined.

You MUST use the ISAM RTE way of initialisation, that is the only way that will work for Federated Repostitories with Basic users !
You MUST have set the bind-dn and bind-pwd under [bind-credentials] in ldap.conf .

Following snippet shows how to correctly initialize the UserLookupHelper, in this case for a username that is submitted in a form.

var username = context.get(Scope.REQUEST, "urn:ibm:security:asf:request:parameter", "username");

var userLookupHelper = new UserLookupHelper();

// Use ISAM RTE with false

// configure bind-dn and bind-pw in ldap.conf

// need to have all certificates in the keystore that is used in ldap.conf (to ad and to rte)

// set 636 as non-ssl port

// MUST use false, because this is the only way Basic Users and Federated repositories will work

// check here
https://www-01.ibm.com/support/docview.wss?uid=ibm10884160
userLookupHelper.init();

reportmsg("Init ... [" + username + "]",false);

if( userLookupHelper.isReady() ) {

  IDMappingExtUtils.traceString("User Lookup Helper is ready for  ... [" + username + "]");

  var user = userLookupHelper.getUser(username);

  ...



But this likely does not work for you.

Enable tracing


When you start working on your InfoMap with UserLookupHelper, you probably want to enable tracing .
Go to "Secure Access Control" , "Global Settings" , "Runtime Parameters", and go the "Runtime Tracing" tab

Image:The only correct way to setup the ISAM RTE with basic users for the UserLookupHelper
Add this trace specification:

com.tivoli.pd.rgy.*=ALL:com.tivoli.am.fim.trustserver.sts.utilities.IDMappingExtUtils=ALL



The trace.log you can then view in "Monitor / Application Logs" (https://lmi/isam/application_logs)  
Under access_control/runtime/trace.log

Errors indication the problem


If you use Basic users in LDAP directories that require SSL, and you have setup your ISAM with local Policy Server/local LDAP, you will likely see errors like this :
The runtime trace shows that the connection uses :389:readwrite:5.

Other errors may look like this:

[7/31/19 11:48:41:962 CEST] 000000bd id=00000000 com.ibm.security.access.user.UserLookupHelper                I getUser com.tivoli.pd.rgy.exception.DomainNotFoundRgyException: HPDAA0266E   The Security Access Manager domain Default does not exist.


or another typical one:

com.tivoli.pd.rgy.exception.ServerDownRgyException: HPDAA0278E   None of the configured LDAP servers of the appropriate type for the operation can be contacted.



Manually configuring ISAM


So how do I fix this ?
The root cause information is correct in the technote, but it does not fully explain how to fix the problem.
Changing the port is only part of the solution, because the embedded LDAP server listens on port 636 of the management interface of the appliance by default, not on localhost. The administrator can choose a port other than the default by modifying the advanced tuning parameter wga.rte.embedded.ldap.ssl.port. The advanced tuning parameters are accessed through Manage System Settings > Advanced Tuning Parameters. After you modify this advanced tuning parameter, you must restart the Security Access Manager runtime environment for the change to take effect.

Update ldap.conf


Go to "Secure Web Settings/Manage/Runtime Component" and navigate to "Manage/Configuration Files/ldap.conf"
Set the host to the ip address or hostname of your LMI, it cannot be set to 127.0.0.1.
Set the port to the same value as ssl-port (by default, 636.  To change it, see the previous paragraph).

[ldap]

host = 192.168.18.250

port = 636

ssl-port = 636


ssl-keyfile = yourkeystore.kdb



The keystore you set here must contain all the CA certificates for the federated ldap.  It's likely you have a value there already if you configured Federated Directories already.
In the default setup, you can export the server certificate from the keystore embedded_ldap_keys , and import it as a CA in "yourkeystore.kdb" .

Update the bind-credentials, with the username and password for a user that can access the embedded ldap (eg. cn=root)

[bind-credentials]

bind-dn = cn=root,secAuthority=Default

bind-pwd = your-root-password-i-hope-it-is-not-passw0rd



The ISAM runtime will restart after these changes.

Update pd.conf


Go to "Secure Web Settings/Manage/Runtime Component" and navigate to "Manage/Configuration Files/pd.conf"

Edit the pdrte stanza.


[pdrte]

user-reg-type = ldap

user-reg-server =

user-reg-host = 192.168.18.250

user-reg-hostport = 636



Update user-reg-server with your lmi's hostname .  Note that you may have to add a hostrecord for your lmi !
Update user-reg-host with your lmi's public ip address (in this example, 192.168.18.250)
Update user-reg-hostport to match the ssl port (default 636).

In the ssl stanza, I enabled the TLS 1.1 and 1.2 versions.


[ssl]

tls-v10-enable = no

tls-v11-enable = yes

tls-v12-enable = yes



Update ivmgrd.conf



Go to "Secure Web Settings/Manage/Runtime Component" and navigate to "Manage/Configuration Files/ivmgrd.conf"

Enable ssl, and assign the keystore you used earlier.

ssl-enabled = true

ssl-keyfile = yourkeystore.kdb




Update your reverse proxies


The reverse proxies need to be instructed to connect over ssl.

So for every reverse proxy, open the "Secure Web Settings"/"Reverse proxy" page, select the reverse proxies one by one and open the Configuration file (webseald.conf): "Manage/Configuration/Edit Configuration file"

Find the [ldap] stanza and enable ssl :

[ldap]

ssl-enabled = yes

ssl-keyfile = yourkeystore.kdb

ssl-keyfile-dn =




The keystore you select here should contain the CA certificates for your embedded ldap , and the federated repositories.  
Leave the ssl-keyfile-dn empty, unless you're using Mutual TLS.

Restart the reverse proxy after these changes, and repeat for all your reverse proxies.

If the reverse proxy fails to start, look in the msg__webseald-.log file (via Monitor/Reverse proxy log files) for errors indicating connection problems, for instance these:

2019-08-01-14:43:48.521+02:00I----- 0x16B480C9 webseald ERROR rgy ira ira_handle.c 1489 0x7f95100a6840 -- HPDRG0201E   Error code 0x51 was received from the LDAP server. Error text: "Can't contact LDAP server".

206       2019-08-01-14:43:48.525+02:00I----- 0x1354A0C0 webseald WARNING ivc general azn_maint.cpp 1136 0x7f94f1b8a700 -- HPDCO0192W   LDAP server 192.168.18.250:636 has failed.



Possible causes are missing ssl certificates , or errors in ip addresses and hostnames in your configuration.

Look for messages like this:
javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.h: No trusted certificate found
javax.naming.CommunicationException: simple bind failed: 192.168.18.250:636 [Root exception is javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.h: No trusted certificate

Ansible configuration


I've published Ansible playbook that performs all these tasks for you https://github.com/Bozzie4/isam_sample_playbooks  
It requires you have setup Ansible to work with ISAM (with the roles and the modules found here :
- https://github.com/IBM-Security/ibmsecurity
- https://github.com/IBM-Security/isam-ansible-roles  

Congratulations, now you should have a working environment


You should now have a working environment :
- you can use a standard pkmslogin to webseal with a basic user
- the UserLookupHelper works correctly (init(false) or init()) and can locate basic users
- the Username/Password mechanism can be configured
- the SCIM setup can use the ISAM RTE

A remark on Point of Contact configuration


This setup is applicable if you have your ISAM appliance set to "Access Manager Credential" or "Access Manager Username and extended attributes".  It is not applicable for the External Users model (since you probably won't have basic users configured ).

A remark LMI Configuration


I noticed that if I set the LMI to only use TLSv1.2, the UserLookupHelper no longer works (or rather, that the ISAM RTE connection fails on an LMIRest call ).  So I re-enabled all TLS protocols for the LMI to resolve this.
I need to get to the bottom of this, though .




Alternative solution that will only work if your federated repositories do not use SSL


If your federated repositories do not use SSL (which will never happen in real life, I hope), you can get away with a simpler change.   However, I do not recommend this

So keep the runtime configured with local Policy Server and local LDAP.

Open ldap.conf, and remove the ssl_keyfile parameter completely.  This will disable all SSL usage for your embedded ldap and for the federated repositories.

Restart everything.