What’s new in OpenDJ 3.0, Part III

FR_plogo_org_FC_openDJ-300x86In the previous posts, I talked about the new PDB Backend in OpenDJ 3.0, and the other changes with backends, replication and the changelog.

In this last article about OpenDJ 3.0, I’m presenting the most important new features and enhancements in this major release:

Certificate Matching Rules.

OpenDJ now implements the CertificateExactMatch matching rule in compliance with “Lightweight Directory Access Protocol (LDAP) Schema Definitions for X.509 Certificates” (RFC 4523) and implements the schema and the syntax for certificates, certificate lists  and certificate pairs.

It’s now possible to search a directory to find an entry with a specific certificate, using a filter such as below:

(userCertificate={ serialNumber 13233831500277100508, issuer rdnSequence:"CN=Babs Jensen,OU=Product Development,L=Cupertino,C=US" })

Password Storage Schemes

The PKCS5S2 Password Storage Scheme has been added to the list of supported storage schemes. While this one is less secure and flexible than PBKDF2, it allows some of our customers to migrate from systems that use the PKCS5S2 algorithm. Other password storage schemes have been enhanced to support arbitrary salt length and thus helping with other migrations (without requiring all users to have a new password).

Disk Space Monitoring.

In previous releases, each backend had a disk space monitoring function, regardless of the filesystems or disks used. In OpenDJ 3.0, we’ve created a disk space monitoring service, and backends, replication, log services register to it. This allows the server to optimise its resource consumption to monitor, as well as ensuring that all disks that contain writable data are monitored, and alerts raised when reaching some low threshold.

Improvements

There are many improvements in many areas of the server: in the REST to LDAP services and gateway, optimisations on indexes, dsconfig batch mode, DSML Gateway supporting SOAP 1.2, native packages… For the complete details, please read the Release Notes.

As always, the best way to really see and feel the difference is by downloading and installing the OpenDJ server, and playing with it. We’re providing a Zip installation, an RPM and a Debian Package, the DSML Gateway and the REST to LDAP Gateway as war files.

Over the course of the development of OpenDJ 3.0, we’ve received many contributions, in form of code, issues raised in our JIRA, documentation… We address our deepest thanks to all the contributors and developers :

Andrea Stani, Auke Schrijnen, Ayami Tyndal, Brad Tumy, Bruno Lavit, Bernhard Thalmayr, Carole Forel, Chris Clifton, Chris Drake, Chris Ridd, Christian Ohr, Christophe Sovant, Cyril Grosjean, Darin Perusich, David Goldsmith, Dennis Demarco, Edan Idzerda, Emidio Stani, Fabio Pistolesi, Gaétan Boismal, Gary Williams, Gene Hirayama, Hakon Steinø, Ian Packer, Jaak Pruulmann-Vengerfeldt, James Phillpotts, Jeff Blaine, Jean-Noël Rouvignac, Jens Elkner, Jonathan Thomas, Kevin Fahy, Lana Frost, Lee Trujillo, Li Run, Ludovic Poitou, Manuel Gaupp, Mark Craig, Mark De Reeper, Markus Schulz, Matthew Swift, Matt Miller, Muzzol Oliba, Nicolas Capponi, Nicolas Labrot, Ondrej Fuchsik, Patrick Diligent, Peter Major, Quentin Cassel, Richard Kolb, Robert Wapshott, Sébastien Bertholet, Shariq Faruqi, Stein Myrseth, Sunil Raju, Tomasz Jędrzejewski, Travis Papp, Tsoi Hong, Violette Roche-Montané, Wajih Ahmed, Warren Strange, Yannick Lecaillez. (I’m sorry if I missed anyone…)

What’s new in OpenDJ 3.0 – Part II

FR_plogo_org_FC_openDJ-300x86Yesterday, I’ve talked about the most important change in OpenDJ 3.0, that is the new PDB Backend. Let me detail other new and improved features of OpenDJ 3.0, still related to backends and replication.

As part of the work for the new backend, we’ve worked on the import process, in order to make it more I/O efficient and thus faster.

Here’s some numbers, importing 1 000 000 users in OpenDJ.

In OpenDJ 2.6.3:

$ import-ldif -l ../1M.ldif -n userRoot
[03/Feb/2016:15:41:42 +0100] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381717 msg=Installation Directory: /Space/Tests/Blog/2.6/opendj
...
[03/Feb/2016:15:42:54 +0100] category=JEB severity=NOTICE msgID=8847454 msg=Processed 1000002 entries, imported 1000002, skipped 0, rejected 0 and migrated 0 in 71 seconds (average rate 13952.5/sec)

In OpenDJ 3.0, with the JE Backend:

$ import-ldif -l ../../1M.ldif -n userRoot
[03/02/2016:15:45:19 +0100] category=UTIL seq=0 severity=INFO msg=Installation Directory: /Space/Tests/Blog/3.0/opendj
...
[03/02/2016:15:46:22 +0100] category=PLUGGABLE seq=74 severity=INFO msg=Processed 1000002 entries, imported 1000002, skipped 0, rejected 0 and migrated 0 in 62 seconds (average rate 15961.2/sec)

In OpenDJ 3.0, with the PDB Backend

$ import-ldif -l ../../1M.ldif -n userRoot
[03/02/2016:15:59:38 +0100] category=UTIL seq=0 severity=INFO msg=Installation Directory: /Space/Tests/Blog/3.0/opendj
...
[03/02/2016:16:00:38 +0100] category=PLUGGABLE seq=48 severity=INFO msg=Processed 1000002 entries, imported 1000002, skipped 0, rejected 0 and migrated 0 in 58 seconds (average rate 17038.7/sec)

We’ve also completely reworked the storage layer for the replication changes, moving away from the BDB JE database. Instead, we’re using direct files, again providing much smaller disk occupancy (and thus optimising I/Os) but also allowing much more efficient purging of old data.

As part of these changes, we’ve made serious improvements to the way the replication changes can be read and searched using LDAP under the “cn=Changelog” suffix. More importantly, we’ve now have a way to ensure a complete ordering of the changes published, and thus consistency of their “changeNumbers”. That is to say that now, when reading “cn=Changelog” on different replicated servers, the change with “ChangeNumber=N” will be the same on all servers, allowing applications that read these changes to failover from one server to another. We’ve added a way to resynchronise these ChangeNumbers when adding a new replica to an existing topology, or when restoring one after a maintenance period.

Still on the subject of the ChangeLog, we’ve added another level of security to it, by introducing a “changelog-read” privilege that provides a better control on which applications and users are allowed to read the data from the “cn=Changelog” suffix.

That’s all for today. Tomorrow, I will continue with all the other new features and enhancements in OpenDJ 3.0.

If you have done it yet, you can download OpenDJ 3.0 from ForgeRock’s BackStage and start playing with it. And check the Release Notes for more information.

OpenDJ 3.0.0 has been released…

FR_plogo_org_FC_openDJ-300x86As part of the release of the ForgeRock Identity Platform that we did last week, we’ve released a major version of our Directory Services product : OpenDJ 3.0.0.

The main and most important change in OpenDJ 3.0 is the work on the backend layer, with the introduction of a new backend database, supported by a new low level key-value store. When installing a new instance of OpenDJ, administrators now have the choice of creating a JE Backend (which is based on Berkeley DB Java Edition, as with previous releases of OpenDJ), or a PDB Backend (which is based on the new PersistIt library). When upgrading, the existing local backends will be transparently upgraded in JE Backends, but indexes will need to be rebuilt (and can be rebuilt automatically during the upgrade process).

Both backends have the same capabilities, and very similar performances. Most importantly, both backends benefit from a number of improvements compared with previous releases : the size of databases and index records are smaller, some indexes have been reworked to deliver better performances both for updates and reads. Overall, we’ve been increasing the throughput of Adding/Deleting entries in OpenDJ by more than 15 %.

But the 2 backends are different, especially in the way they deal with database compression. Because of the way it’s dealing with journals and compression, the new PDB backend may deliver better overall throughput, but may increase its disk occupancy significantly under heavy load (it favours updates over compression). Once the throughput is reduced under a certain threshold, compression will be highly effective and the overall disk occupancy will be optimised.

A question I often get is “Which backend should I use? “. And I don’t have a definitive answer. If you have an OpenDJ instance and you’re upgrading to 3.0, keep the JE Backend. This is a simple and automated upgrade. If you’re installing a new instance of OpenDJ, then I would say it’s a matter of risks. We don’t have the same wide experience with the PDB backend than we have had with the JE backend over the last 10 years. So, if you want to be really safe, chose the JE Backend. If you have time to test, stage your directory service before putting it in production, you might want to go with the PDB Backend. As, moving forward, we will focus our performance testing and improvements on the PDB backend essentially.

That’s all for now. In a followup post, I will continue to review the changes in OpenDJ 3.0…

Meanwhile, you can download OpenDJ 3.0 from ForgeRock’s BackStage and start playing with it. And check the Release Notes for more information.

PS: The followup posts have been published:

New version of ForgeRock Identity Platform™

This week, we have announced the release of the new version of the ForgeRock Identity Platform, which brings new services in the following areas :

  • Continuous Security at Scale
  • Security for Internet of Things (IoT)
  • Enhanced Data Privacy Controls

FRPlatform

This is also the first identity management solution to fully implement the User-Managed Access (UMA) standard, making it possible for organizations to address expanding privacy regulations and establish trusted digital relationships. See the article that Eve Maler, VP of Innovation at ForgeRock and Chief UMAnitarian posted to explain UMA and what it can do for you.

A more in depth description of the new features of the ForgeRock Identity Platform has also been posted.

The ForgeRock Identity Platform is available for download now at https://www.forgerock.com/downloads/

In future posts, I will detail what is new in the Directory Services part, built on the OpenDJ project.

Nouvelle version de la Plateforme Identité de ForgeRock

Cette semaine nous venons d’annoncer la nouvelle version de la Plateforme d’Identité de ForgeRock (ForgeRock Identity Platform™).

FRPlatform

La Plateforme d’Identité de ForgeRock est maintenant capable d’évaluer dans son contexte et en continu, l’authenticité des utilisateurs, des appareils et des objets.

Cette nouvelle version est aussi la première solution qui offre le support de la norme “User Managed Access” (UMA) qui permet aux individus de partager, contrôler, autoriser et révoquer l’accès aux données de façon sélective, et donc offrent aux entreprises une solution ouverte et standardisée pour protéger et contrôler la confidentialité des données de leurs clients et employés. Ces besoins de confidentialité et de gestion du consentement deviennent importants dans le domaine de la santé, des objets connectés ou même dans le secteur des services financiers.

Pour mieux comprendre “UMA” et les services offerts par la Plateforme d’Identité de ForgeRock, je vous propose de regarder cette courte vidéo (en Anglais).

La plateforme ForgeRock Identity Platform est disponible en téléchargement dès à présent à l’adresse : https://www.forgerock.com/downloads/

Les détails des nouveautés de cette version sont sur le site de ForgeRock.

LDAPCon 2015

22494196563_56cdbd5a6c_zTime flies… LDAPCon 2015 has happened and we all have returned from Edinburgh to our daily lives.

As for the previous editions, this year’s conference was well attended, very friendly, with plenty of time to socialize around a (few) pint(s) of beer.

23126811911_71434b0311_mDavid Goodman started the conference with a keynote presentation that illustrated the long path followed by LDAP, but also how important it still is in the major industries, especially in the Telco world.

My 2 presentations were given on the first day of the conference. The first one was about “LDAP Asynchronous Programming” and the Promises API we’ve added in the OpenDJ LDAP SDK.

The second presentation was an update on the OpenDJ project with a highlight on what is in the OpenDJ 3.0 release due mid December.

All of the presentations are already available through the web site, either in the Programme section, or directly in the Downloads section.

Thanks and kudos to this year’s organisers : Andrew Findlay and Stephen Quinney.

As usual, you can get a glimpse of the conference and people on my photo album.

LDAPCon 2015 photo album by Ludovic Poitou
LDAPCon 2015