directory

About LDAP Syntaxes and backward compatibility…

Ludo4 Comments

In the LDAP information model, a syntax constrains the structure and format of attribute values. OpenDJ defines and implements a large number of syntaxes (you can discover them by reading the ldapSyntaxes attribute from the cn=Schema entry).

But infrequently, we receive enquiries on an obscure and non standard syntax, often in the form of “I’m having an error importing schema from this or that legacy directory server”, with an error message that ends with “No such syntax is configured for use in the Directory Server”.

As syntaxes are constraining the structure and format of attribute values, they are implemented as code, specifically Java code in OpenDJ. It’s possible to implement new syntaxes by implementing the org.opends.server.api.AttributeSyntax abstract class, and installing the classes or the JAR in OpenDJ classpath. But often, it’s easier and more convenient to define a syntax by configuration, and OpenDJ offers 3 possibilities to define new syntaxes. In term of backward compatibility, I will only focus on the 2 main ones, by substitution and by pattern (the 3rd one allows to define enumeration of values).

With OpenDJ, you can define a new syntax by configuration and delegating the contraints to an already implemented syntax. A simple example is the URI syntax (which was defined is some very old schema with the OID  1.3.6.1.4.1.4401.1.1.1). A URI is really an ASCII string, and it might be sufficient to accept attributes with URI syntax to verify that all characters are pure ASCII. The standard syntax for ASCII strings is IA5String aka 1.3.6.1.4.1.1466.115.121.1.15.

ldapSyntaxes: ( 1.3.6.1.4.1.4401.1.1.1 DESC ‘URI’ X-SUBST ‘1.3.6.1.4.1.1466.115.121.1.15’ )

Insert the above line in the schema LDIF file before the attributeTypes, and you’re done.

The other option is to define the syntax as a pattern, using regular expressions. This could be better when willing to enforce additional constraints on an URI, for example, verifying that the URI is an LDAP one.

ldapSyntaxes: ( 999.999.999.1 DESC 'LDAP URI Syntax' X-PATTERN '^ldap://[-a-zA-Z0-9+&@#/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#/%=~_|]' )

So the next time you are trying to import some legacy schema to the OpenDJ directory server, and you have an error due to missing syntaxes, you know what to do to quickly resolve the problem.

LDAPCon 2013 – a summary…

Ludo1 Comment

ldapcon_2013_logo_line_dateLast Monday and Tuesday (Nov 18-19), I was in Paris attending the 4th International LDAP Conference, an event I help to organize with LDAPGTF, a network of French actors in the LDAP and Identity space. ForgeRock was also one of the 3 gold sponsors of the conference along with Symas and Linagora.

LDAPCon 2013The conference happens every other year and is usually organized by volunteers from the community. This year, the French guys were the most motivated, especially Clément Oudot from Linagora, leader of the LDAP Tool Box and lemonLDAP projects, and Emmanuel Lecharny one of the most active developers on Apache Directory Server.

I was honored to be the keynote and first speaker of the conference and presented “The Shift to Identity Relationship Management“, which was well received and raised a lot of interest from the audience.

The first day was focusing more on the users of LDAP and directory services technologies, and several presentations were made about REST interfaces to directory services, including the standard in progress: SCIM.

Kirian Ayyagari, from the Apache Directory project, presented his work on SCIM and the eSCIMo project. Present for the first time at LDAPCon, Microsoft’s  Philippe Beraud spoke about Windows Azure Active Directory and its Graph API. And I talked about and demoed the REST to LDAP service that we’ve built in OpenDJ. For the demo, I used PostMan, a test client for HTTP and APIs, but also our newly open sourced sample application for Android : OpenDJ contact manager. In the afternoon, Peter Gietz talked about the work he did around SPML and SCIM leveraging OpenLDAP access log.

After many talks about REST, we had a series of talk around RBAC. Shawn McKinney presented the Fortress open source IAM project and more specifically the new work being done around RBAC. Then Peter, Shawn and Markus Widmer talked about the effort to build a common LDAP schema for RBAC. And Matthew Hardin talked about the OpenLDAP RBAC overlay bringing policy decisions within the directory  when deploying Fortress.

Then followed presentations about local directory proxy services for security based on OpenLDAP, about Red Hat FreeIPA (another first appearance at LDAPCon) and about OpenLDAP configuration management with Apache Directory Studio. Also Stefan Fabel came all the way from Hawaii ( Aloha ! ) to present a directory based application for managing and reporting publications by a university: an interesting story about building directory schema and data model.

The day ended with a presentation from Clement Oudot about OpenLDAP and the password policy overlay. As usual, talking about the LDAP password policy internet-draft raises the question of when it will be finally published as an RFC. While there is a consensus that it’s important to have a standard reference document for it, I’m failing to see how we can dedicate resources to achieve that goal. Let’s see if someone will stand up and take the leadership on that project.

After such a long day of talks and discussion, most of the attendees converged to a nearby pub where we enjoyed beers and food while winding down the day through endless discussions.

The second day of LDAPCon 2013 was more focused on developers and the development of directory services. It was a mix of status and presentations of open source directory projects like OpenDJ, OpenLDAP or LSC, some discussions about backend services, performance design considerations and benchmarks, a talk about Spring LDAP… As usual, we had a little bit of a musical introduction to Howard Chu‘s presentation.

LP0_1068I enjoyed the Benchmark presentation by Jillian Kozyra, which was lively, rational and outlining the major difference between open source based products and closed source ones (although all closed source products were anonymized due to license restrictions). It’s worth noting that Jillian is pretty new in the directory space and she seems to have tried to be as fair as possible with her tests, but she did say that the best documented product and the easiest one to install and deploy is OpenDJ. Yeah !!! 🙂

Another interesting talk was Christian Hollstein‘s about his “Distributed Virtual Transaction Directory Server“, a telco grade project he’s working on to serve the needs of the 4G network services (such as HSS, HLR…). It’s clear to me that telco operators and network equipment providers are now all converging to LDAP technologies for the network and this drives a lot of requirements on the products (something I knew since we started the OpenDS project at Sun, kept in mind while developing OpenDJ, even though right now our focus has mainly been on the large enterprises and consumer facing directory services).

All the slides of the conference have been made available online through the LDAPCon.org website and the Lanyrd event page. Audio has also been recorded and will be made available once processed. And as usual, all the photos that I took during the conference are publicly available in my Flickr LDAPCon 2013 Set. Feel free to copy for personal use.

It’s been a great edition of the LDAPCon and I’m looking forward to the next one, in 2 years !

Meanwhile I’d like to thanks the sponsors, all 75 attendees, the 19th speakers and the 2 organizers I had not mentioned yet : M.C. Jonathan Clarke and Benoit Mortier.

Updates to opendj-utils

Ludo2 Comments

About a year ago, I’ve introduced a set of OpenDJ scripts and utilities that I’ve built to facilitate my work with OpenDJ.

Last week, I’ve pushed some updates to the github repository.

The first improvements are in logstat.py. I’ve added support for collecting stats about the Abandon operation, as well as some counting and reports on the errors to each operation. This allows to get a feel of how many operations failed and the error code reported.

The second update is a new utility names filterstat.py which scans through access log files and builds a sorted list of all filters used in search requests. The filters are generalized and collated together, and the result should help administrators to understand which attributes should be indexed and what kind of index are required.

Here’s a sample output of filterstat (based on an instance of OpenDJ used by OpenAM). The first value is the count and the string is the generic representation of the filter:

$ ~/opendj-utils/filterstat.py access
processing file: access
213783 (&(uid=VALUE)(objectclass=VALUE))
2080 (&(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)))
807 (&(objectclass=VALUE)(uniqueMember=VALUE))
244 (&(cn=VALUE)(objectclass=VALUE))
213 (&(&(uid=VALUE)(objectclass=VALUE)))
140 (&(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)))
63 (&(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)))
39 (&(&(cn=VALUE)(objectclass=VALUE)))
6 (&(uid=*SUBSTRING*)(objectclass=VALUE))
2 (&(objectclass=VALUE)(ou=VALUE))
1 (&(&(uid=*)(objectclass=VALUE))(|(uid=VALUE)))
Base search filters only:
2487 (|(objectclass=*)(objectclass=VALUE))
Done

Please give those tools a try and let me know how useful they are for you. And if you have ideas on how to improve them, feel free to fork them and contribute.

OpenDJ 2.6.0 is now available

Ludo2 Comments

OpenDJ-300x100I am really happy to announce the general availability of OpenDJ 2.6.0, a major update of ForgeRock  directory service product, built from the tag 2.6.0 (revision 9086 in our SVN repository).

OpenDJ 2.6.0 brings a lot of added value, including :

– A REST to LDAP service, allowing an easy access to directory data using HTTP/JSON. The service can be run either embedded in the server or as a standalone web application.

– A new upgrade process to ease transition from OpenDJ 2.4.5 or newer to 2.6.

– New Linux native packages (RPM and Debian) to facilitate the automatic deployment of OpenDJ in the private and public cloud.

– OpenDJ can be configured to delegate authentication to a Microsoft Active Directory service, providing tighter integration with Microsoft environment without the burden of synchronizing passwords.

– An optional extension to remove specific attributes from updates, making it more flexible and easier to deal with legacy applications and migration tasks.

– A way to synchronize SAMBA password attributes with the user’s password.

– Some improvements on the integrity of references, that is now enforced at creation or on update.

– More flexible and efficient audit logs.

– A Java based LDAP software development kit.

– An official stable documentation.

For the complete list of new features, enhancements and fixed defects, please read the release notes.

The binaries can be downloaded from ForgeRock Downloads.

Over the course of the development of OpenDJ, we’ve received many contributions, in form of code, issues raised in our JIRA, documentation… We address our deepest thanks to all the contributors and developers :

Aiman Tahboub, Alan Evans, Arturo V Sanchez, Auke Schrijnen, Bernhard Thalmayr, Brent Palmer, Bruno Vernay, Chris Dowey, Chris Ridd, Christophe Sovant, Dan Gardner, Danny Turner, Darin Perusich, Donal Duane, Elliot Kendall, Eswar Moorthy, Fred Voss, Gael Allioux, Gary Williams, German Parente, Göran Odmyr, Ian McGlothlin, Jamie Nelson, Jean-Noël Rouvignac, Jeff Blaine, Jeffrey Crawford, Jens Elkner, Lana Frost, Laurent Bristiel, Ludovic Poitou, Manuel Gaupp, Manuel Schallar, Mark Craig, Mark Gibson, Marko Harjula, Martin Sperle, Matthew Stevenson, Matthew Swift, Miroslav Fadrhonc, Mitch Silverstein, Nemanja Lukić, Nicholas Sushkin , Nikolay Belaevski, Per-Olov Sjoholm, Peter Major, Rauli Ikonen, Sachiko Wallace, Slavomir Katuscak, Tomas Forsman, Vanessa Richie, Violette Roche, Willi Burmeister

Happy 4th of July everyone !

Tips: Do not index virtual attributes in OpenDJ

LudoLeave a comment

OpenDJ-300x100OpenDJ, the open source LDAP directory service in Java, offer some interesting services to reduce and optimize the size and usage of data.

One of them is the Virtual Attribute feature, which allow certain attributes and values to be computed as needed, either based on some of the server internals or other attributes. OpenDJ ships with a number of virtual attributes by default : entryDN, entryUUID, etag, gouverningStructureRule, hasSubordinate, isMemberOf, numSubordinate, password Expiration Time (ds-pwp-password-expiration-time), structuralObjectClass, subSchemaSubEntry, …

Since these attributes are virtual and thus not stored as part of the entries in the database backend, you must not define any index for them. When possible, the virtual attribute provider will make use of default system index (like entryDN uses the DN index), but most of the time, these attributes are for reading and consuming.

If you do configure an index for one of the virtual attribute, the server will repeatedly report that the index is degraded with an error message similar to the following :

[21/Jan/2013:09:16:07 +0000] category=JEB severity=NOTICE msgID=8847510 msg=Due to changes in the configuration, index dc_example_dc_com_entryDN is currently operating in a degraded state and must be rebuilt before it can be used

And then some seaches may fail to return entries. So you must delete this index to let the server behave properly.

Upcoming events: LavaJUG & Devoxx France

Ludo2 Comments

I will be at the LavaJUG (Java User Group from Clermont-Ferrand, France) this Thursday from 19:00 to 21:00, presenting our experience with the OpenDJ project with building a highly scalable and high performance server in Java. The presentation is based on what I’ve already presented in a few JUG in France (AlpesJUG, MarsJUG, PoitouCharentesJUG,…) and Switzerland (JUG Lausanne), but has been updated with regards to GarbageFirst GC and the most recent HotSpot JVM.

 

And next week, from  March Wednesday 27th to Friday 29th, you will find ForgeRock at the Devoxx France conference.

Come to our conference session about “Enterprise Security in a Cloudy and Mobile World” (the session is in French). The session is on Friday 29th, from 11:45 to 12:35, in Miles Davis room. Mark it on your calendar, and if you miss it, make sure you stop by our booth (B3) to say hello and talk with some of our engineers. We will also be present at the HackerGarten on Wednesday from 14:00 to 18:00, should you want to have fun with one of our open source projects : OpenAM, OpenDJ or OpenIDM.

DevoxxFR-2013-banniere-texte-600-232

OpenDJ scripts and utilities on github

Ludo1 Comment

Following the exemple of my colleague Chris, I’ve pushed a set of OpenDJ scripts and utilities for that I’ve wrote and used in the past to github.

You will find script for log statistics, converting schema, …

Feel free to use, and if you have suggested enhancements, patches, I’ll be looking for pull request !

OpenDJ 2.4.6 is now available

LudoLeave a comment

As few days after an important milestone for OpenDJ, the open source LDAP directory server in Java, I’m happy to announce that a new bug fix release of  the 2.4 series has just been made available. OpenDJ 2.4.6 is an update release of the OpenDJ project and improves reliability and performances with large groups and entries, as well as very large databases. The full details about the release have been posted in the OpenDJ 2.4.6 Release Notes. Upgrading to this release is recommended for everyone running earlier versions. For additional features and bug fixes, please use OpenDJ 2.5.0-Xpress1.

The release is built out of revision 8102 of the b2.4 branch of the code repository.

As usual, you can find every thing on the OpenDJ Downloads page:

The draft documentation for OpenDJ, and more specifically the Administration Guide, has been updated on the OpenDJ project site, still on the track for an accurate, reviewed version for OpenDJ 2.5.

Feedback is important to us and you can participate on the IRC channel, the mailing lists or join our community.

Enjoy !

OpenDJ 2.5.0-Xpress1 is now available

Ludo3 Comments

I’m happy to announce that a new revision of OpenDJ, the open source LDAP directory server in Java has just been released. OpenDJ 2.5.0-Xpress1 is a new stable release of the main development branch of the OpenDJ project.

OpenDJ 2.5.0-Xpress1 brings you the latest features such as:

  • Capability to delegate authentication to Microsoft Active Directory (pass-through authentication)
  • Improved enforcement of referential integrity for groups, whereby OpenDJ can now ensure both that members’ entries exist when they are added to groups, and also that members are removed from groups when their entries are deleted
  • Access log filtering, with additional output configuration to combine request and response messages, log control OIDs, and specify timestamp formats
  • Optimistic concurrency control through ETag attributes
  • Synchronization of Samba and OpenDJ passwords

You can find more details about the OpenDJ 2.5.0-Xpress1 release in the OpenDJ Release Notes.

The release is built out of revision 8087 of the trunk of the code repository.

As usual, you can find every thing on the OpenDJ Downloads page:

The draft documentation for OpenDJ, and more specifically the Administration Guide, has been updated on the OpenDJ project site, still on the track for an accurate, reviewed version for the final release of OpenDJ 2.5.0, due by the end of this year.

Feedback is important to us and you can participate on the IRC channel, the mailing lists or join our community.

Enjoy !

Assigning a Custom Password policy to a subTree

Ludo4 Comments

OpenDJ supports defining password policies that are quite complete in term of security measures to reduce the risks associated with textual passwords. It also defines 2 default policies, one for the administrators such as “cn=Directory Manager”, and one for all other users : the “Default Password Policy”. But it is possible to define additional password policies and assign them to individual users or group of users. Today, we are considering how to assign a password policy to all users under a specific subtree. In the article below, I first define a new custom password policy and then I demonstrate 2 ways of assigning that password policy to all persons under the ou=people,dc=example,dc=com subtree.

Defining a custom password policy using dsconfig:

$ dsconfig create-password-policy \
 --set default-password-storage-scheme:Salted\ SHA-256 \
 --set password-attribute:userpassword \
 --type generic \
 --policy-name Custom\ PP \
 --hostname lpmac.local \
 --port 4444 \
 --bindDN cn=Directory\ Manager \
 --bindPassword ****** \
 -X -n

1- Assigning the password policy through a Virtual Attribute.

$ dsconfig create-virtual-attribute \
 --set attribute-type:ds-pwp-password-policy-dn \
 --set enabled:true \
 --set value:cn=Custom\ PP,cn=Password\ Policies,cn=config \
 --set base-dn:ou=people,dc=example,dc=com \
 --set filter:\(objectClass=person\) \
 --type user-defined \
 --name Custom\ PP\ Assignment \
 --hostname lpmac.local \
 --port 4444 \
 --bindDN cn=Directory\ Manager \
 --bindPassword ****** \
 -X -n

Check that the password policy is assigned properly:

$ ldapsearch -D "cn=directory manager" -w secret12 -p 1389 -b "" 'uid=user.1' '+' userPassword
dn: uid=user.1,ou=People,dc=example,dc=com
userPassword: {SSHA}u+52Ld6iaTvFoNlQvqTHrn1BBW9IjjT2/I25hg==
numSubordinates: 0
ds-pwp-password-policy-dn: cn=Custom PP,cn=Password Policies,cn=config
structuralObjectClass: inetOrgPerson
pwdPolicySubentry: cn=Custom PP,cn=Password Policies,cn=config
subschemaSubentry: cn=schema
hasSubordinates: false
entryDN: uid=user.1,ou=people,dc=example,dc=com
entryUUID: 4e9b7847-edcb-3791-b11b-7505f4a55af4

Change the user password, the new password should be encoded with the scheme specified (SSHA-256)

$ ldappasswordmodify -p 1389 -D uid=user.1,ou=People,dc=example,dc=com -w password -A -n newPassword
 The LDAP password modify operation was successful

$ ldapsearch -D "cn=directory manager" -w secret12 -p 1389 -b "" 'uid=user.1' userPassword
dn: uid=user.1,ou=People,dc=example,dc=com
userPassword: {SSHA256}vjIdZEtF1AIiM0EgY9unZUXXublwQwlOCoe4RYEIHtpzumW1hYyvNg==

2 – Assigning the password policy using Collective Attributes :

$ ldapmodify -D cn=directory\ manager -w secret12 -p 1389
dn: cn=Pwp for Users,dc=example,dc=com
changetype: add
objectclass: collectiveAttributeSubEntry
objectclass: extensibleObject
objectclass: subentry
objectclass: top
ds-pwp-password-policy-dn;collective: cn=Custom PP,cn=Password Policies,cn=config
subtreeSpecification: { base "ou=people", specificationFilter "(objectclass=person)"}

Processing ADD request for cn=Pwp for Users,dc=example,dc=com
ADD operation successful for DN cn=Pwp for Users,dc=example,dc=com

Now we can check that the password policy is well assigned, and that it’s used when changing password for example.

$ ldapsearch -D "cn=directory manager" -w secret12 -p 1389 -b "" 'uid=user.1' '+' userPassword
dn: uid=user.1,ou=People,dc=example,dc=com
userPassword: {SSHA}6tHBLHh2C25UpAsKX0eq0d6LEXYGX+Jcm4dh7g==
numSubordinates: 0
ds-pwp-password-policy-dn: cn=Custom PP,cn=Password Policies,cn=config
structuralObjectClass: inetOrgPerson
etag: 000000008211ac6a
pwdPolicySubentry: cn=Custom PP,cn=Password Policies,cn=config
subschemaSubentry: cn=schema
hasSubordinates: false
collectiveAttributeSubentries: cn=Pwp for Users,dc=example,dc=com
entryDN: uid=user.1,ou=people,dc=example,dc=com
entryUUID: 4e9b7847-edcb-3791-b11b-7505f4a55af4

$ ldappasswordmodify -p 1389 -D uid=user.1,ou=People,dc=example,dc=com -w password -A -n newPassword
 The LDAP password modify operation was successful

$ ldapsearch -D "cn=directory manager" -w secret12 -p 1389 -b "" 'uid=user.1' userPassword
 dn: uid=user.1,ou=People,dc=example,dc=com
 userPassword: {SSHA256}WswyH9ANoKcxQWlSn/eL8h/dNk532K/e5zGlJcwiwMLsCQqw+cAX0Q==

So which method to assign a password policy to specific users is best ?

The first method should be preferred when the password policy is defined in the configuration (as we’ve done in the example). Both configuration entries, the password policy and its assignment, are under the “cn=config” tree,  but need to be defined in all replicas.

The second method defines the assignment of a policy to users as an subentry collocated with the data, and will be replicated. It should be preferred if the password policy is also defined as a subEntry, along with its assignment. Such way of configuring a password policy is documented in the Administration Guide, Configuring Password Policies section, procedure 10.3 – To Create a Subentry Based Password Policy.

More secure passwords !

Ludo8 Comments

I’ve received an intriguing request from a customer last week :  he wanted to know if we’ve done benchmarks of the password hashing schemes that are available in OpenDJ, our LDAP directory service. Their fear was that with stronger schemes, they could not sustain a high authentication rate.

In light of the LinkedIn leak of several millions of passwords, hashed with a simple unsalted SHA1, I decided to run a quick and simple test.

SSHA1 is the default hashing scheme for password in OpenDJ. The salt is an 8 bytes (64-bit) random string and is used with the password to produce the 20 bytes message digest. But OpenDJ directory server supports a wide range of password hashing scheme and salted SHA512 is currently the most secure hashing algorithm we support (and the salt here is also an 8 bytes (64-bit) random octet string).

So for the test, I generated a sample directory data set with 10 000 users, and imported it in the OpenDJ directory (a 2.5 development build) with the default settings, on my laptop (MacBook Pro, 2.2 GHz intel Core i7).

$ ldapsearch -D "cn=directory manager" -w secret12 -p 1389 -b "dc=example,dc=com" 'uid=user.10' dn userPassword
dn: uid=user.10,ou=People,dc=example,dc=com
userPassword: {SSHA}cchzM+LrPCvbZdthOC8e62d4h7a4CfoNvl6d/w==

I then ran an “authrate” which is a small benchmark tool that allows to stress an LDAP server with a high number of authentications (LDAP Bind requests) and let it run to 5 minutes.

authrate -h localhost -p 1389 -g 'rand(0,10000)' -D "uid=user.%d,ou=people,dc=example,dc=com" -w password -c 32 -f
-----------------------------------------------------------------
 Throughput     Response Time
 (ops/second)   (milliseconds)
 recent average recent average 99.9% 99.99% 99.999% err/sec
 -----------------------------------------------------------------
 ...
 26558.0  26148.9   1.179    1.195  10.168  19.431  156.421      0.0

I then stopped the server, changed the import default password encryption scheme to Salted SHA512, and reimported the data.

$ ldapsearch -D "cn=directory manager" -w secret12 -p 1389 -b "dc=example,dc=com" 'uid=user.10' dn userPassword
 dn: uid=user.10,ou=People,dc=example,dc=com
 userPassword: {SSHA512}eTGiwtTM4niUKNkEBy/9t03UdbsyYTL1ZXhy6uFnw4X0T6Y9Zf5/dS7hDIdx3/UTlUQ/9JjNV9fOg2BkmVgBhWWu5WpWKPog

And then re-run the “authrate”

$ authrate -h localhost -p 1389 -g 'rand(0,10000)' -D "uid=user.,ou=people,dc=example,dc=com" -w password -c 32 -f
 -----------------------------------------------------------------
 Throughput     Response Time
 (ops/second)   (milliseconds)
 recent average recent average 99.9% 99.99% 99.999% err/sec
 -----------------------------------------------------------------
 ...
 25481.7 25377.6 1.222 1.227 10.470 15.473 158.234 0.0

As you can see, there is not much of a difference in throughput or response time, when using the strongest algorithm to hash user password. So do not hesitate to change the default settings and make use of the strongest password hashing schemes with OpenDJ. It could save you from the embarrassment of, one day, contacting each of your users or customers to ask them to change their compromised password.

The default password hashing schemes are in 2 locations :

  • The default password policy for all passwords that are changed online.
dn: cn=Default Password Policy,cn=Password Policies,cn=config
ds-cfg-default-password-storage-scheme: cn=Salted SHA-512,cn=Password Storage Schemes,cn=config
  • In the Import Password Policy
dn: cn=Password Policy Import,cn=Plugins,cn=config
ds-cfg-default-user-password-storage-scheme: cn=Salted SHA-512,cn=Password Storage Schemes,cn=config

Both properties can be changed with dsconfig while the OpenDJ server is running, and the new scheme will be used for all subsequent operations.

An Optimized solution for directory services ?

Ludo4 Comments

I was recently pointed to a white paper published by Oracle called : Oracle Optimized Solution for Oracle Unified Directory — Implementation Guide.

Because Oracle Unified Directory and OpenDJ have a common root (both derive from the Sun initiated OpenDS project), I was curious about that optimized solution, and if there was anything that might be applicable for our customers. And after reading the 45 pages white-paper, honestly, the Oracle Optimized Solution is not something I would recommend to any of our customers (*).

The white paper describes the hardware used for the solution : 3 SPARC T4-1 systems with 128GB of RAM and 6 300GB internal disks each. A Sparc T4-1 machine is an 8 core machine with each core supporting up to 8 threads. Each T4-1 system has 10GbE add-on network card. And each T4-1 machine is attached to a Sun Storage 2500-M2 array (with 2540 controllers) through two fiber channel cards, and each storage has 12 disks.

Let’s see the average price for this solution : The SPARC T4-1 with 128 GB of RAM has an estimated public price of $24,344, but with only 2 internal disks. So add another $1,660 for the additional 4 disks. The lowest price for a 10GbE card for that system is $2,000 and the cheapest storage array with the same amount of disks roughly $27,000. A total cost per system over $55,000, not including the cost of the Operating System, and a total cost for the “Optimized Solution” of approximately $165,000 (estimated public price).

So what do we get in performance for this price ? Well the white paper will not tell you what the solution is optimized for. The only number that appears, is the time it took to import the 15 Million entries on one of the systems :

[07/Jan/2012:12:19:29 +0000] category=JEB severity=NOTICE msgID=8847569 msg=Total import time was 3790 seconds. Phase one processing completed in 2868 seconds, phase two processing completed in 922 seconds
[07/Jan/2012:12:19:29 +0000] category=JEB severity=NOTICE msgID=8847454 msg=Processed 15000001 entries, imported 15000001, skipped 0, rejected 0 and migrated 0 in 3790 seconds (average rate 3957.0/sec)
[07/Jan/2012:12:19:29 +0000] category=JEB severity=NOTICE msgID=8847536 msg=Import LDIF environment close took 0 seconds

Last week, I was in Mexico with a partner of ours, demonstrating the capabilities of OpenDJ with the customer’s data (exported from a week ago, and which also contains several hundreds of very large static groups). We used x.86 based machines, with 96GB of memory, although we only used 16GB for the instance of OpenDJ.

And here’s the output of the import command :

[25/Apr/2012:20:10:44 +0200] category=JEB severity=NOTICE msgID=8847538 msg=DN phase two processing completed. Processed 21654508 DNs
[25/Apr/2012:20:10:45 +0200] category=JEB severity=NOTICE msgID=8847569 msg=Total import time was 2002 seconds. Phase one processing completed in 1137 seconds, phase two processing completed in 865 seconds
[25/Apr/2012:20:10:45 +0200] category=JEB severity=NOTICE msgID=8847454 msg=Processed 21654508 entries, imported 21654508, skipped 0, rejected 0 and migrated 0 in 2002 seconds (average rate 10815.3/sec)
[25/Apr/2012:20:10:45 +0200] category=JEB severity=NOTICE msgID=8847536 msg=Import LDIF environment close took 0 seconds

I don’t have the price for the servers we used (but our partner can get in touch with you if you’re interested in the solution), but I doubt that it tops half of the price of the Oracle optimized solution !

So before you drink the Oracle cool-aid, think twice about what an optimized solution should be, and how much it should cost. Oh, by the way, there is no cost in license for OpenDJ, it’s open source, it’s available now and you can try it free of charge. Of course, we do appreciate if you subscribe to one of our support offering to protect your investment and ensure some Service Level Agreement.

(*) I would not recommend a directory solution on SPARC Tx machine ever. While the machines have a good capacity for load, the performance for any write activity is really bad, especially as soon as access controls are in use. Most of our partners who have been deploying directory services on these machines will agree with me. As a matter of fact, I don’t recall any recent customer mentioning SPARC nor Solaris when renewing their directory service infrastructure.

A timeline of LDAP directory services…

Ludo1 Comment

Bill Nelson,  has published the “The Most Complete History of Directory Services You Will Ever Find” (until the next one comes along), a detailed history of LDAP based directory services and products. Expect a few updates as people find about this and ask for adding new data points. But this is the most complete summary I’m aware of. I had a timeline of Sun directory products a few years ago, but Bill’s has more details.

His post includes a visual timeline of the directory service products and their heritage, linked here under, for your convenience.

Click on the picture for a full size image.

Personally, I’ve been involved with the Sun and derived lines since 1996, and now drive the ForgeRock one: OpenDJ !

Tab Sweep for Friday April 13th

LudoLeave a comment

Another week goes by, and it’s time for another tab sweep.

Syntegrity Networks, one of our major partners in the US, has launched a campaign to encourage their customers to migrate from Sun Directory Server to OpenDJ.

Silverpeas, a Collaborative Platform, built as open source under the GNU Affero license by the eponym company, has been supporting LDAP for authentication and authorization for some time. The documentation for setting up the LDAP domain has been updated using OpenDJ as the recommended server.

ForgeRock OpenIDM capabilities are growing. After getting OpenIDM to work with Activity to provide workflows, the team posted a experimental tutorial to integrate Jasper with OpenIDM to produce nice reports. You can find more of these tutorials in the OpenIDM How To Collection.

Tips: resource limits in OpenDJ

Ludo4 Comments
Photo by Scallop Holden http://www.flickr.com/photos/scallop_holden/
Photo by Scallop Holden http://www.flickr.com/photos/scallop_holden/

OpenDJ, the open source LDAP directory services in Java, defines a few global resource limits to prevent client connections or operations from abusing the server’s resources. These limits are

  • the maximum number of entries returned to a search request (size-limit, default is 1000),
  • the maximum amount of time to spend returning results to a client (time-limit, default is 60 seconds),
  • the maximum number of entries to look through while processing a search request (lookthrough-limit, default is 5000),
  • the maximum amount of time a connection can sit idle before the server disconnect it (idle-time-limit, default is unlimited).

There are default values for all of these limits in the Global configuration, but they can also be set on a per user basis. The global limits are read or set using dsconfig :

$ bin/dsconfig get-global-configuration-prop -p 4444 -X -n -h localhost \
 -D cn=directory\ manager -w secret12
Property : Value(s)
--------------------------------------:------------------------
bind-with-dn-requires-password : true
default-password-policy : Default Password Policy
disabled-privilege : -
entry-cache-preload : false
etime-resolution : milliseconds
idle-time-limit : 0
lookthrough-limit : 5000
max-allowed-client-connections : 0
max-psearches : unlimited
proxied-authorization-identity-mapper : Exact Match
reject-unauthenticated-requests : false
return-bind-error-messages : false
save-config-on-successful-startup : true
size-limit : 1000
smtp-server : -
time-limit : 60 s
writability-mode : enabled

The per user limits have a different LDAP attribute name and can be found or set directly in users’ entry, or through Collective Attributes. The Directory Manager entry has such specific limits set, so that everything is unlimited.

$ bin/ldapsearch -D "cn=directory manager" -w secret12 -p 1389 -X -b "cn=config" \
  '(objectClass=inetOrgPerson)' ds-rlim-time-limit ds-rlim-size-limit \
  ds-rlim-lookthrough-limit ds-rlim-idle-time-limit
dn: cn=Directory Manager,cn=Root DNs,cn=config
ds-rlim-lookthrough-limit: 0
ds-rlim-time-limit: 0
ds-rlim-idle-time-limit: 0
ds-rlim-size-limit: 0

If you decide to change the default global settings, for example the idle-time-limit, to force idle connections to be closed by the server after some time (often a smaller time than the settings of the load-balancer in between your applications and the OpenDJ servers), please remember that you might also want to change the limit for “cn=Directory Manager”, especially if your client applications are connecting with Directory Manager credentials.