What’s new in OpenDJ 3.0, Part III

FR_plogo_org_FC_openDJ-300x86In the previous posts, I talked about the new PDB Backend in OpenDJ 3.0, and the other changes with backends, replication and the changelog.

In this last article about OpenDJ 3.0, I’m presenting the most important new features and enhancements in this major release:

Certificate Matching Rules.

OpenDJ now implements the CertificateExactMatch matching rule in compliance with “Lightweight Directory Access Protocol (LDAP) Schema Definitions for X.509 Certificates” (RFC 4523) and implements the schema and the syntax for certificates, certificate lists  and certificate pairs.

It’s now possible to search a directory to find an entry with a specific certificate, using a filter such as below:

(userCertificate={ serialNumber 13233831500277100508, issuer rdnSequence:"CN=Babs Jensen,OU=Product Development,L=Cupertino,C=US" })

Password Storage Schemes

The PKCS5S2 Password Storage Scheme has been added to the list of supported storage schemes. While this one is less secure and flexible than PBKDF2, it allows some of our customers to migrate from systems that use the PKCS5S2 algorithm. Other password storage schemes have been enhanced to support arbitrary salt length and thus helping with other migrations (without requiring all users to have a new password).

Disk Space Monitoring.

In previous releases, each backend had a disk space monitoring function, regardless of the filesystems or disks used. In OpenDJ 3.0, we’ve created a disk space monitoring service, and backends, replication, log services register to it. This allows the server to optimise its resource consumption to monitor, as well as ensuring that all disks that contain writable data are monitored, and alerts raised when reaching some low threshold.


There are many improvements in many areas of the server: in the REST to LDAP services and gateway, optimisations on indexes, dsconfig batch mode, DSML Gateway supporting SOAP 1.2, native packages… For the complete details, please read the Release Notes.

As always, the best way to really see and feel the difference is by downloading and installing the OpenDJ server, and playing with it. We’re providing a Zip installation, an RPM and a Debian Package, the DSML Gateway and the REST to LDAP Gateway as war files.

Over the course of the development of OpenDJ 3.0, we’ve received many contributions, in form of code, issues raised in our JIRA, documentation… We address our deepest thanks to all the contributors and developers :

Andrea Stani, Auke Schrijnen, Ayami Tyndal, Brad Tumy, Bruno Lavit, Bernhard Thalmayr, Carole Forel, Chris Clifton, Chris Drake, Chris Ridd, Christian Ohr, Christophe Sovant, Cyril Grosjean, Darin Perusich, David Goldsmith, Dennis Demarco, Edan Idzerda, Emidio Stani, Fabio Pistolesi, Gaétan Boismal, Gary Williams, Gene Hirayama, Hakon Steinø, Ian Packer, Jaak Pruulmann-Vengerfeldt, James Phillpotts, Jeff Blaine, Jean-Noël Rouvignac, Jens Elkner, Jonathan Thomas, Kevin Fahy, Lana Frost, Lee Trujillo, Li Run, Ludovic Poitou, Manuel Gaupp, Mark Craig, Mark De Reeper, Markus Schulz, Matthew Swift, Matt Miller, Muzzol Oliba, Nicolas Capponi, Nicolas Labrot, Ondrej Fuchsik, Patrick Diligent, Peter Major, Quentin Cassel, Richard Kolb, Robert Wapshott, Sébastien Bertholet, Shariq Faruqi, Stein Myrseth, Sunil Raju, Tomasz Jędrzejewski, Travis Papp, Tsoi Hong, Violette Roche-Montané, Wajih Ahmed, Warren Strange, Yannick Lecaillez. (I’m sorry if I missed anyone…)

What’s new in OpenDJ 3.0 – Part II

FR_plogo_org_FC_openDJ-300x86Yesterday, I’ve talked about the most important change in OpenDJ 3.0, that is the new PDB Backend. Let me detail other new and improved features of OpenDJ 3.0, still related to backends and replication.

As part of the work for the new backend, we’ve worked on the import process, in order to make it more I/O efficient and thus faster.

Here’s some numbers, importing 1 000 000 users in OpenDJ.

In OpenDJ 2.6.3:

$ import-ldif -l ../1M.ldif -n userRoot
[03/Feb/2016:15:41:42 +0100] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381717 msg=Installation Directory: /Space/Tests/Blog/2.6/opendj
[03/Feb/2016:15:42:54 +0100] category=JEB severity=NOTICE msgID=8847454 msg=Processed 1000002 entries, imported 1000002, skipped 0, rejected 0 and migrated 0 in 71 seconds (average rate 13952.5/sec)

In OpenDJ 3.0, with the JE Backend:

$ import-ldif -l ../../1M.ldif -n userRoot
[03/02/2016:15:45:19 +0100] category=UTIL seq=0 severity=INFO msg=Installation Directory: /Space/Tests/Blog/3.0/opendj
[03/02/2016:15:46:22 +0100] category=PLUGGABLE seq=74 severity=INFO msg=Processed 1000002 entries, imported 1000002, skipped 0, rejected 0 and migrated 0 in 62 seconds (average rate 15961.2/sec)

In OpenDJ 3.0, with the PDB Backend

$ import-ldif -l ../../1M.ldif -n userRoot
[03/02/2016:15:59:38 +0100] category=UTIL seq=0 severity=INFO msg=Installation Directory: /Space/Tests/Blog/3.0/opendj
[03/02/2016:16:00:38 +0100] category=PLUGGABLE seq=48 severity=INFO msg=Processed 1000002 entries, imported 1000002, skipped 0, rejected 0 and migrated 0 in 58 seconds (average rate 17038.7/sec)

We’ve also completely reworked the storage layer for the replication changes, moving away from the BDB JE database. Instead, we’re using direct files, again providing much smaller disk occupancy (and thus optimising I/Os) but also allowing much more efficient purging of old data.

As part of these changes, we’ve made serious improvements to the way the replication changes can be read and searched using LDAP under the “cn=Changelog” suffix. More importantly, we’ve now have a way to ensure a complete ordering of the changes published, and thus consistency of their “changeNumbers”. That is to say that now, when reading “cn=Changelog” on different replicated servers, the change with “ChangeNumber=N” will be the same on all servers, allowing applications that read these changes to failover from one server to another. We’ve added a way to resynchronise these ChangeNumbers when adding a new replica to an existing topology, or when restoring one after a maintenance period.

Still on the subject of the ChangeLog, we’ve added another level of security to it, by introducing a “changelog-read” privilege that provides a better control on which applications and users are allowed to read the data from the “cn=Changelog” suffix.

That’s all for today. Tomorrow, I will continue with all the other new features and enhancements in OpenDJ 3.0.

If you have done it yet, you can download OpenDJ 3.0 from ForgeRock’s BackStage and start playing with it. And check the Release Notes for more information.

OpenDJ 3.0.0 has been released…

FR_plogo_org_FC_openDJ-300x86As part of the release of the ForgeRock Identity Platform that we did last week, we’ve released a major version of our Directory Services product : OpenDJ 3.0.0.

The main and most important change in OpenDJ 3.0 is the work on the backend layer, with the introduction of a new backend database, supported by a new low level key-value store. When installing a new instance of OpenDJ, administrators now have the choice of creating a JE Backend (which is based on Berkeley DB Java Edition, as with previous releases of OpenDJ), or a PDB Backend (which is based on the new PersistIt library). When upgrading, the existing local backends will be transparently upgraded in JE Backends, but indexes will need to be rebuilt (and can be rebuilt automatically during the upgrade process).

Both backends have the same capabilities, and very similar performances. Most importantly, both backends benefit from a number of improvements compared with previous releases : the size of databases and index records are smaller, some indexes have been reworked to deliver better performances both for updates and reads. Overall, we’ve been increasing the throughput of Adding/Deleting entries in OpenDJ by more than 15 %.

But the 2 backends are different, especially in the way they deal with database compression. Because of the way it’s dealing with journals and compression, the new PDB backend may deliver better overall throughput, but may increase its disk occupancy significantly under heavy load (it favours updates over compression). Once the throughput is reduced under a certain threshold, compression will be highly effective and the overall disk occupancy will be optimised.

A question I often get is “Which backend should I use? “. And I don’t have a definitive answer. If you have an OpenDJ instance and you’re upgrading to 3.0, keep the JE Backend. This is a simple and automated upgrade. If you’re installing a new instance of OpenDJ, then I would say it’s a matter of risks. We don’t have the same wide experience with the PDB backend than we have had with the JE backend over the last 10 years. So, if you want to be really safe, chose the JE Backend. If you have time to test, stage your directory service before putting it in production, you might want to go with the PDB Backend. As, moving forward, we will focus our performance testing and improvements on the PDB backend essentially.

That’s all for now. In a followup post, I will continue to review the changes in OpenDJ 3.0…

Meanwhile, you can download OpenDJ 3.0 from ForgeRock’s BackStage and start playing with it. And check the Release Notes for more information.

PS: The followup posts have been published:

New version of ForgeRock Identity Platform™

This week, we have announced the release of the new version of the ForgeRock Identity Platform, which brings new services in the following areas :

  • Continuous Security at Scale
  • Security for Internet of Things (IoT)
  • Enhanced Data Privacy Controls


This is also the first identity management solution to fully implement the User-Managed Access (UMA) standard, making it possible for organizations to address expanding privacy regulations and establish trusted digital relationships. See the article that Eve Maler, VP of Innovation at ForgeRock and Chief UMAnitarian posted to explain UMA and what it can do for you.

A more in depth description of the new features of the ForgeRock Identity Platform has also been posted.

The ForgeRock Identity Platform is available for download now at https://www.forgerock.com/downloads/

In future posts, I will detail what is new in the Directory Services part, built on the OpenDJ project.

Nouvelle version de la Plateforme Identité de ForgeRock

Cette semaine nous venons d’annoncer la nouvelle version de la Plateforme d’Identité de ForgeRock (ForgeRock Identity Platform™).


La Plateforme d’Identité de ForgeRock est maintenant capable d’évaluer dans son contexte et en continu, l’authenticité des utilisateurs, des appareils et des objets.

Cette nouvelle version est aussi la première solution qui offre le support de la norme “User Managed Access” (UMA) qui permet aux individus de partager, contrôler, autoriser et révoquer l’accès aux données de façon sélective, et donc offrent aux entreprises une solution ouverte et standardisée pour protéger et contrôler la confidentialité des données de leurs clients et employés. Ces besoins de confidentialité et de gestion du consentement deviennent importants dans le domaine de la santé, des objets connectés ou même dans le secteur des services financiers.

Pour mieux comprendre “UMA” et les services offerts par la Plateforme d’Identité de ForgeRock, je vous propose de regarder cette courte vidéo (en Anglais).

La plateforme ForgeRock Identity Platform est disponible en téléchargement dès à présent à l’adresse : https://www.forgerock.com/downloads/

Les détails des nouveautés de cette version sont sur le site de ForgeRock.

OpenIG 3.1 is now available…

It’s my great pleasure to announce the general availability of OpenIG 3.1, a minor update of the ForgeRock Open Identity Gateway product, following the press release of early December.

The Open Identity Gateway is a simple standard-based solution to secure access to web applications and APIs. It supports SAMLv2, OAuth 2.0, OpenID Connect and can capture and replay credentials, enabling SSO and Federation.

With a four months release cycle since the previous release, OpenIG 3.1 doesn’t contain many major new features, but it does bring several new enhancements to the product, including :

  • The support for encrypted JSON Web Token (JWT) cookies to store session information on the user-agent. The administrator can decide to keep the default container managed sessions, or use JWT cookies globally or for a specific route.
  • A simplification of OpenIG configuration, with the ability to inline objects, omit specific fields when empty or obvious. This simplification enables faster configuration as well as a better readability for long term maintenance of the service.
  • IMG_4090The introduction of “Decorator” for configuration objects, easily adding new behaviors to existing configured objects. OpenIG 3.1 provides 3 decorators out of the box: a CaptureDecorator that enables debugging and logging in a much easier and more dynamic way; a TimerDecorator that records times spent in the decorated objects; an AuditDecorator that allows to audit operations for any decorated objects.
  • The support for a sample monitoring handler that provides basic statistics about the exchanges and routes. The monitoring information can be used to provide an activity dashboard such as here on the right..
  • Some optimisations and performance improvements when using OpenID Connect and OAuth 2.0

For the complete details of the changes in OpenIG 3.1, please check the release notes.

You can download the ForgeRock product here. It’s been heavily tested by our Quality Assurance team : functional tests on Windows, Mac and Linux, stress tests as proxy, with OAuth2 and OpenID Connect, non-regression tests… The documentation has been entirely reviewed and all examples tested.  The  source code is available in our code repository (https://svn.forgerock.org/openig).

We are interested in your feedback, so get it, play with it and give us your comments, either on the mailing list, the wiki, the OpenIG Forum or through blog posts.


On track for the release…

Yesterday we’ve announced the new releases of 3 of ForgeRock products : OpenAM 12.0, OpenIDM 3.1 and OpenIG 3.1.

There are still a few days before we make OpenIG 3.1 generally available. We are currently stressing it out and reviewing the documentation. But all indicators are green for now :

Snapshot of OpenIG Issue Tracker