Data Confidentiality with OpenDJ LDAP Directory Services

FR_plogo_org_FC_openDJ-300x86Directory Servers have been used and continue to be used to store and retrieve identity information, including some data that is sensitive and should be protected. OpenDJ LDAP Directory Services, like many directory servers, has an extensive set of features to protect the data, from securing network connections and communications, authenticating users, to access controls and privileges… However, in the last few years, the way LDAP directory services have been deployed and managed has changed significantly, as they are moving to the “Cloud”. Already many of ForgeRock customers are deploying OpenDJ servers on Amazon or MS Azure, and the requirements for data confidentiality are increasing, especially as the file system and disk management are no longer under their control. For that reason, we’ve recently introduced a new feature in OpenDJ, giving the ability to administrators to encrypt all or part of the directory data before writing to disk.clouddataprotection

The OpenDJ Data Confidentiality feature can be enabled on a per database backend basis to encrypt LDAP entries before being stored to disk. Optionally, indexes can also be protected, individually. An administrator may chose to protect all indexes, or only a few of them, those that contain data that should remain confidential, like cn (common name), sn (surname)… Additionally, the confidentiality of the replication logs can be enabled, and then it’s enabled for all changes of all database backends. Note that if data confidentiality is enabled on an equality index, this index can no longer be used for ordering, and thus for initial substring nor sorted requests.

Example of command to enable data confidentiality for the userRoot backend:

dsconfig set-backend-prop \
 -h opendj.example.com -p 4444 \
 -D "cn=Directory Manager" -w secret12 -n -X \
 --backend-name userRoot --set confidentiality-enabled:true

Data confidentiality is a dynamic feature, and can be enabled, disabled without stopping the server. When enabling on a backend, only the updated or created entries will be encrypted. If there is existing data that need confidentiality, it is better to export and reimport the data. With indexes data confidentiality, the behaviour is different. When changing the data confidentiality on an index, you must rebuild the index before it can be used with search requests.

Key Management - Photo adapted from https://www.flickr.com/people/ecossystems/

When enabling data confidentiality, you can select the cipher algorithm and the key length, and again this can be per database backend. The encryption key itself is generated on the server itself and securely distributed to all replicated servers through the replication of the Admin Backend (“cn=admin data”), and thus it’s never exposed to any administrator. Should a key get compromised, we provide a way to mark it so and generate a new key. Also, a backup of an encrypted database backend can be restored on any server with the same configuration, as long as the server still has its configuration and its Admin backend intact. Restoring such backend backup to fresh new server requires that it’s configured for replication first.

The Data Confidentiality feature can be tested with the OpenDJ nightly builds. It is also available to ForgeRock customers as part of our latest update of the ForgeRock Identity Platform.

OpenDJ LDAP Directory Services update

FR_plogo_org_FC_openDJ-300x86The new version of ForgeRock Directory Services, based on OpenDJ 3.0, was released in January and I’ve already written about the new features here, here and here.

We’ve now started the development of the next releases. We’ve updated the high level roadmap on our wiki, to give you an idea of what’s coming.

The last few weeks have been very active, as you can see on our JIRA dashboard.

Screen Shot 2016-03-18 at 10.56.12

There are already a few new features and enhancements in the master branch of our GIT repository :

A Bcrypt password storage scheme. The new scheme is meant to help migration of user accounts from other systems, without requiring a password reset. Bcrypt also provide a much stronger level of security for hashing passwords, as it’s number of iteration is configurable. But since OpenDJ 2.6, we are already providing a PBKDF2 password storage scheme which is recommended over Bcrypt by OWASP, for securing passwords.

Some enhancements of our performance testing tools, part of the OpenDJ LDAP Toolkit. All xxxxrate tools have a new way of computing statistics, providing more reliable and consistent results while reducing the overhead of producing them.

Some performance enhancements in various areas, including replication, group management, overall requests processing…

If you want to see it by yourself, you can checkout the code from our GIT repository, and build it, or you can grab the latest nightly build.

Play with OpenDJ and let us know how it works for you.

What’s new in OpenDJ 3.0, Part III

FR_plogo_org_FC_openDJ-300x86In the previous posts, I talked about the new PDB Backend in OpenDJ 3.0, and the other changes with backends, replication and the changelog.

In this last article about OpenDJ 3.0, I’m presenting the most important new features and enhancements in this major release:

Certificate Matching Rules.

OpenDJ now implements the CertificateExactMatch matching rule in compliance with “Lightweight Directory Access Protocol (LDAP) Schema Definitions for X.509 Certificates” (RFC 4523) and implements the schema and the syntax for certificates, certificate lists  and certificate pairs.

It’s now possible to search a directory to find an entry with a specific certificate, using a filter such as below:

(userCertificate={ serialNumber 13233831500277100508, issuer rdnSequence:"CN=Babs Jensen,OU=Product Development,L=Cupertino,C=US" })

Password Storage Schemes

The PKCS5S2 Password Storage Scheme has been added to the list of supported storage schemes. While this one is less secure and flexible than PBKDF2, it allows some of our customers to migrate from systems that use the PKCS5S2 algorithm. Other password storage schemes have been enhanced to support arbitrary salt length and thus helping with other migrations (without requiring all users to have a new password).

Disk Space Monitoring.

In previous releases, each backend had a disk space monitoring function, regardless of the filesystems or disks used. In OpenDJ 3.0, we’ve created a disk space monitoring service, and backends, replication, log services register to it. This allows the server to optimise its resource consumption to monitor, as well as ensuring that all disks that contain writable data are monitored, and alerts raised when reaching some low threshold.

Improvements

There are many improvements in many areas of the server: in the REST to LDAP services and gateway, optimisations on indexes, dsconfig batch mode, DSML Gateway supporting SOAP 1.2, native packages… For the complete details, please read the Release Notes.

As always, the best way to really see and feel the difference is by downloading and installing the OpenDJ server, and playing with it. We’re providing a Zip installation, an RPM and a Debian Package, the DSML Gateway and the REST to LDAP Gateway as war files.

Over the course of the development of OpenDJ 3.0, we’ve received many contributions, in form of code, issues raised in our JIRA, documentation… We address our deepest thanks to all the contributors and developers :

Andrea Stani, Auke Schrijnen, Ayami Tyndal, Brad Tumy, Bruno Lavit, Bernhard Thalmayr, Carole Forel, Chris Clifton, Chris Drake, Chris Ridd, Christian Ohr, Christophe Sovant, Cyril Grosjean, Darin Perusich, David Goldsmith, Dennis Demarco, Edan Idzerda, Emidio Stani, Fabio Pistolesi, Gaétan Boismal, Gary Williams, Gene Hirayama, Hakon Steinø, Ian Packer, Jaak Pruulmann-Vengerfeldt, James Phillpotts, Jeff Blaine, Jean-Noël Rouvignac, Jens Elkner, Jonathan Thomas, Kevin Fahy, Lana Frost, Lee Trujillo, Li Run, Ludovic Poitou, Manuel Gaupp, Mark Craig, Mark De Reeper, Markus Schulz, Matthew Swift, Matt Miller, Muzzol Oliba, Nicolas Capponi, Nicolas Labrot, Ondrej Fuchsik, Patrick Diligent, Peter Major, Quentin Cassel, Richard Kolb, Robert Wapshott, Sébastien Bertholet, Shariq Faruqi, Stein Myrseth, Sunil Raju, Tomasz Jędrzejewski, Travis Papp, Tsoi Hong, Violette Roche-Montané, Wajih Ahmed, Warren Strange, Yannick Lecaillez. (I’m sorry if I missed anyone…)

OpenDJ 3.0.0 has been released…

FR_plogo_org_FC_openDJ-300x86As part of the release of the ForgeRock Identity Platform that we did last week, we’ve released a major version of our Directory Services product : OpenDJ 3.0.0.

The main and most important change in OpenDJ 3.0 is the work on the backend layer, with the introduction of a new backend database, supported by a new low level key-value store. When installing a new instance of OpenDJ, administrators now have the choice of creating a JE Backend (which is based on Berkeley DB Java Edition, as with previous releases of OpenDJ), or a PDB Backend (which is based on the new PersistIt library). When upgrading, the existing local backends will be transparently upgraded in JE Backends, but indexes will need to be rebuilt (and can be rebuilt automatically during the upgrade process).

Both backends have the same capabilities, and very similar performances. Most importantly, both backends benefit from a number of improvements compared with previous releases : the size of databases and index records are smaller, some indexes have been reworked to deliver better performances both for updates and reads. Overall, we’ve been increasing the throughput of Adding/Deleting entries in OpenDJ by more than 15 %.

But the 2 backends are different, especially in the way they deal with database compression. Because of the way it’s dealing with journals and compression, the new PDB backend may deliver better overall throughput, but may increase its disk occupancy significantly under heavy load (it favours updates over compression). Once the throughput is reduced under a certain threshold, compression will be highly effective and the overall disk occupancy will be optimised.

A question I often get is “Which backend should I use? “. And I don’t have a definitive answer. If you have an OpenDJ instance and you’re upgrading to 3.0, keep the JE Backend. This is a simple and automated upgrade. If you’re installing a new instance of OpenDJ, then I would say it’s a matter of risks. We don’t have the same wide experience with the PDB backend than we have had with the JE backend over the last 10 years. So, if you want to be really safe, chose the JE Backend. If you have time to test, stage your directory service before putting it in production, you might want to go with the PDB Backend. As, moving forward, we will focus our performance testing and improvements on the PDB backend essentially.

That’s all for now. In a followup post, I will continue to review the changes in OpenDJ 3.0…

Meanwhile, you can download OpenDJ 3.0 from ForgeRock’s BackStage and start playing with it. And check the Release Notes for more information.

PS: The followup posts have been published:

OpenDJ Nightly Builds…

For the last few months, there’s been a lot of changes in the OpenDJ project in order to prepare the next major release : OpenDJ 3.0.0. While doing so, we’ve tried to keep options opened and continued to make most of the changes in the trunk/opends part, keeping the possibility to release a 2.8 version. And we’ve made tons of work in branches as well as in trunk/opendj. As part of the move to the trunk, we’ve changed the factory to now build with Maven. Finally, at the end of last week, we’ve made the switch on the nightly builds and are now building what will be OpenDJ 3, from the trunk.

For those who are regularly checking the nightly builds, the biggest change is going to be the version number. The new build is now showing a development version of 3.0.

$ start-ds -V
OpenDJ 3.0.0-SNAPSHOT
Build 20150506012828
--
 Name Build number Revision number
Extension: snmp-mib2605 3.0.0-SNAPSHOT 12206

We are still missing the MSI package (sorry to the Windows users, we are trying to find the Maven plugin that will allow us to build the package in a similar way as previously with ant), and we are also looking at restoring the JNLP based installer, but otherwise OpenDJ 3 nightly builds are available for testing, in different forms : Zip, RPM and Debian packages.

OpenDJ Nightly Builds at ForgeRock.org

We have also changed the minimal version of Java required to run the OpenDJ LDAP directory server. Java 7 or higher is required.

We’re looking forward to getting your feedback.

Why I love my job !

At ForgeRock, I have multiple reasons to enjoy what I do. I have the responsibility for two products: OpenDJ, the LDAP directory services and OpenIG the Identity Gateway, and I also manages the French subsidiary. But what really gets me excited in the morning is that I get to work with very smart and passionate people!

Jean-Noël, one of the engineers of the OpenDJ development team, has a passion for beautiful code and AlpesJuggyTranshe loves refactoring, cleaning existing code. On his personal time, he started to automate his process in Eclipse, and then turn it into an Eclipse plugin, and finally made the code available as an open source project: AutoRefactor. Now, in the office, most of the engineers using Eclipse are also using the AutoRefactor plugin.

So when Jean-Noël got to present his work at our local Java User Group (the AlpesJUG), the rest of the team went along and supported him. As one of the other engineers has a passion for photography (which I share), it gives this amazing picture gallery and set of souvenirs for everyone:

AutoRefactor Session at the AlpesJUG (Feb 24, 2015)
Photos by Bruno Lavit – Click to go to the picture gallery

PS: It also helps that we are working in a great environment where we can afford to do this⬇︎ (sometime to time) during our lunch break!

FondChamrousse

OpenDJ on Windows…

OpenDJ LogoOpenDJ, the LDAP directory services in Java, is supported on multiple platforms and has been for many years. We’re testing on Linux, Windows, Solaris, Mac OS X, but also different JVMs: Oracle JRE, OpenJDK, Azul Zulu, IBM JVM…

With OpenDJ 2.6, we’ve made it easier for people to install it on Linux machines by providing RPM and Debian packages.

We are now also providing a MSI package to ease the installation and removal on Windows machines. The MSI package is available for nightly builds here.

OpenDJ MSI InstallerScreen Shot 2015-01-28 at 09.14.01