The specific final proposal which has arisen from the project is that, starting from the next operating system upgrade, (i.e. the upgrade to Scientific Linux 7), all DICE client machines should be configured as 'conventional' LDAP clients. This discussion document attempts to summarize the pros and cons of that proposal.
"The Lightweight Directory Access Protocol (LDAP) is an open, vendor-neutral, industry standard application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network."
"[LDAP] Directories are usually accessed using the client/server model of communication. An application that wants to read or write information in a directory does not access the directory directly. Instead, it calls a function or application programming interface (API) that causes a message to be sent to another process. This second process accesses the information in the directory on behalf of the requesting application via TCP/IP. ...
"The request is performed by the directory client, and the process that maintains and looks up information in the directory is called the directory server. In general, servers provide a specific service to clients. Sometimes, a server might become the client of other servers in order to gather the information necessary to process a request.
"The client and server processes may or may not be on the same machine. A server is capable of serving many clients. ..."
In a typical Linux deployment, LDAP is used to provide to the client machine the same user data as Yellow Pages / NIS once did: namely
netgroup entries. Access to those is set up via suitable configuration in the
/etc/nsswitch.conf file. Note that this means that data describing 'normal' (i.e. non-system) users to the client system is not available should LDAP lookups fail in any way.
DICE machines use LDAP to look up
netgroup entries describing 'normal' (i.e. non-system) users via suitable configuration in
/etc/nsswitch.conf. Since authorization to access any DICE machine is moderated by these entries, this means that, in the event of an LDAP failure, a DICE machine cannot be logged into as any user other than
In addition, the entire DICE authorization system -
capabilities/entitlements - is arranged via LDAP.
All of this means that LDAP plays a crucial role in the DICE security model; indeed, the role of LDAP in overall security was a deliberate design choice at the time that DICE was initially developed. The corollary is that, whatever might be chosen to replace the current overall LDAP setup, there is a clear requirement that security must be maintained.
As described above, LDAP is designed as a distributed service: the architecture is client/server. In addition, multiple LDAP servers themselves can themselves operate in a federated way: the content of a single LDAP directory information tree can be split between a group of cooperating servers, each of which then returns referrals to clients as and when necessary.
In practice, so far as the authors are aware, most sites deploying LDAP adopt a networked client/server model: client machines make lookups over the network against distinct LDAP server machines.
For its LDAP implementation, DICE uses the OpenLDAP software, along with related open source modules. Currently, every DICE machine - desktop, application server, file server, anything else - runs its own fully-populated LDAP server, and all LDAP lookups on the machine are made against that local server. (More detail on this is provided in Section 3.1 Current client configuration.
Why was this design chosen over the more 'conventional' one described in Section 1.4 How is LDAP generally deployed? above?:
slapddaemon was prone to frequent and apparently random crashes. To limit the effects of problems on a central server affecting all DICE clients, it made sense for each DICE client to run its own local LDAP server.
Comment: OpenLDAP is now much more reliable: we no longer experience random crashes of the
slapd daemon on either the LDAP master or slave servers.
ls -lin a directory of files owned by many different owners was unacceptably slow.)
Comment: Whatever were the exact reasons for that behaviour, recent testing indicates that it is no longer the case.
[Taken from Review of OpenLDAP DICE client configuration, with minor corrections/clarifications]
slapreplruns via SASL->GSS-API->-Kerberos, so the exchange is both authenticated and encrypted. Machines on the 'stable' release currently synchronise against the single master LDAP server; machines on the 'develop' release sychronise against the slaves, via the DNS round-robin entry
getpwnam), that lookup always proceeds via the so-called 'name service ldap connection daemon',
nslcd. The LDAP server therefore has no knowledge of the UID of the actual process which made the lookup request. No Kerberos/SASL authentication or encryption is involved: the LDAP request is done via anonymous bind, and transmitted in plain-text.
nslcd is an integral part of the redesigned
nss-pam-ldapd module (see Arthur de Jong's design notes) and, on DICE, first appeared in the move to SL6. It is not a caching daemon, and there are possible alternatives - in particular, the replacement of
nslcd by the caching daemon
ldapsearchnormally proceeds via SASL and Kerberos, so the exchange is both authenticated and encrypted.
!openldap.server mSET(dir.inf.ed.ac.uk)) then the same authentication/encryption setups apply.
Specifically that means that, on our setup, no authentication or encryption is used for remote (i.e. off-machine) LDAP lookups which are done via the
nslcd daemon. All such lookups are done via anonymous binds and proceed over unencrypted links.
dice-authorizepackage. We need to establish whether or not this is the only such case.
Comment: the exact details of how that robustness is arranged depend on which client caching/connection daemon is in use on the clients. See Appendix A: caching/connection daemons
nslcdor the caching daemon
Comment: which one of these two daemons will be used is yet to be decided - see Appendix A: caching/connection daemons
syncreplmechanism) in order that they remain capable of truly autonomous operation. Such LDAP slaves will provide service to themselves only.
We expect such cases to be rare, but the facility exists.
slaprepl(and any associated code) needs to be maintained, and to be ported at each OS upgrade.
slapdhas (at least in the past) proven to be prone to high memory use; this is possibly inappropriate and unnecessary on 'client' machines.
Note that, at present, we are unaware of any such dependencies.
In practice: we need to be quite sure that - in the event of serious system failures - we are able to do the necessary operations on the servers which control important parts of our overall infrastructure which will to allow us to correct problems which might be affecting the rest of our systems.
dice-authorizesoftware needs to be changed so that it no longer has a hard-coded dependency on
Red Hat Linux (and therefore Scientific Linux) provides two daemons which can be interposed between the LDAP client and server:
nslcd- the Name Service LDAP Connection Daemon - is an integral part of the redesigned
nss-pam-ldapdmodule (see Arthur de Jong's design notes), and first appeared in the move to SL6. It is specifically designed as an interface between LDAP clients and servers; nothing else. It is not a caching daemon.
sssd- the System Security Services Daemon - is a more recent development whose 'primary function is to provide access to identity and authentication remote resource through a common framework' - see https://fedorahosted.org/sssd/. It can be used as an interface between LDAP clients and servers, amongst other things. It does provide caching.
In the SL6 production version of DICE, we use
nscld. We assume that both
sssd daemons will be available in SL7, but we get the impression that the use of
sssd might generally come to be favoured. In the context of this document, the principal advantage of
nscld is its caching functionality.
Testing of the proposed client/server LDAP configuration on CO desktops over the past year has only involved
nslcd. We have now done initial testing with
sssd, and we are fairly confident that it can successfully integrated into a client/server DICE LDAP configuration.
Using either daemon will allow LDAP server failover and redundancy, but the implementation details will differ and, in either case, further work and tuning remains to be done.
In the test client/server LDAP configuration currently running on CO's desktops, the configuration is as follows:
nslcditself copes with individual servers which become responsive.
A similar configuration might be suitable for a final deployment, but we need to check whether or not the DNS round-robin works successfully across all Informatics subnets. If it turns out that subnet-specific tailoring is necessary, then multiple
uri: entries in each client's
nslcd.conf might be a better arrangement. If necessary, load-balancing could be enforced by the randomization (per client) of such entries.
One proposed configuration is as follows:
ldapuristring defined in the
man sssd-ldapfor more detailed information)
ldapuriwould take the form
dir0.inf.ed.ac.uk,dir1.inf.ed.ac.uk,...where we would expect each named server to be a permanent service address.
ldapuristring whenever required - this will be under LCFG control.
As an alternative, we note that
sssd claims to support both load-balancing and failover through DNS service discovery (i.e. SRV records.) This needs to be tested.
In whichever way we might want to deploy
sssd, it will require further rigorous testing.
If we retain our existing LDAP configuration, then all of our existing LDAP servers (which are physical hardware) are due to be replaced in the current financial year. If we choose to move to the proposed configuration, we would need to clarify the following:
None of these are easy questions to answer in advance of any real deployment, and tuning after the fact will almost certainly be necessary. Note, though, that the use of either
sssd means that LDAP servers can be added to (or removed from) the running service 'on the fly'.
Our current preference is to have at least one (and ideally more, for multiple-failure-proofing) physical LDAP servers under our control, in order that dependencies are as clear as possible. Such a pool of physical servers could be topped up with additional VMs, as and when load demands.
There are 'effort' implications to either decision on this matter, but no more detailed estimates have yet been made.
Ideally, any work arising needs to be completed in time for the SL7 deployment. In case of delays, we have the option of falling back on the current setup (provided that that is suitably ported and tested), and transitioning to a new setup at some later date.
Please contact us with any comments or corrections.
Unless explicitly stated otherwise, all material is copyright The University of Edinburgh