Updates to opendj-utils

About a year ago, I’ve introduced a set of OpenDJ scripts and utilities that I’ve built to facilitate my work with OpenDJ.

Last week, I’ve pushed some updates to the github repository.

The first improvements are in logstat.py. I’ve added support for collecting stats about the Abandon operation, as well as some counting and reports on the errors to each operation. This allows to get a feel of how many operations failed and the error code reported.

The second update is a new utility names filterstat.py which scans through access log files and builds a sorted list of all filters used in search requests. The filters are generalized and collated together, and the result should help administrators to understand which attributes should be indexed and what kind of index are required.

Here’s a sample output of filterstat (based on an instance of OpenDJ used by OpenAM). The first value is the count and the string is the generic representation of the filter:

$ ~/opendj-utils/filterstat.py access
processing file: access
213783 (&(uid=VALUE)(objectclass=VALUE))
2080 (&(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)))
807 (&(objectclass=VALUE)(uniqueMember=VALUE))
244 (&(cn=VALUE)(objectclass=VALUE))
213 (&(&(uid=VALUE)(objectclass=VALUE)))
140 (&(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)))
63 (&(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE))(|(sunxmlKeyValue=VALUE)(sunxmlKeyValue=VALUE)))
39 (&(&(cn=VALUE)(objectclass=VALUE)))
6 (&(uid=*SUBSTRING*)(objectclass=VALUE))
2 (&(objectclass=VALUE)(ou=VALUE))
1 (&(&(uid=*)(objectclass=VALUE))(|(uid=VALUE)))
Base search filters only:
2487 (|(objectclass=*)(objectclass=VALUE))
Done

Please give those tools a try and let me know how useful they are for you. And if you have ideas on how to improve them, feel free to fork them and contribute.

What a great ForgeRock European Open Identity Summit !

Chateau BehoustLast week, ForgeRock hosted its first european Open Identity Summit, in the “Chateau de Béhoust” just outside Paris. For two and half days, our 110+ visitors, a mix of customers, prospect customers, partners and consultants, could attend presentations, meet and greet with ForgeRock employees, have lengthy discussions with peers, exchanging experience or use case scenarios around the ForgeRock Open Identity Stack. All of this in a very relaxed and friendly atmosphere.

All of the presentations have been filmed and will be available shortly through our web site and the summit page. If you missed the event and want to get a feel of the content, please check Simon Moffat’s review.

As usual, I’ve taken a few pictures of the event.

Thanks to all attendees and sponsors of the event. And see you next year for the second edition of our ForgeRock summits.

LP0_0488LP0_0538LP0_0595

OpenDJ is Java 8 ready…

Java 8 CompatibleA few weeks ago, I came upon the Adopt openjdk program, launched by the London Java User Group. I’ve decided to give it a try and for that to leverage the Dev@Cloud and FOSS program from CloudBees.

While building OpenDJ I hit a roadblock : XJC is defective with OpenJDK 8 and prevents us from building the DSML gateway that is part of the OpenDJ project. It’s only recently when the openjdk bug database was made publicly available that I found it was a known P1 issue, yet still not resolved.

Recently someone filed a bug against OpenDJ for failing with openjdk 8, so we paid more attention, found the cause of the failure and fixed it. And now OpenDJ directory server is working fine on openjdk 8. We are keeping an automatic build and test with openjdk8 (*) to make sure things will work when Java 8 is released.

Next steps will be to verify that OpenDJ still works with the beta version of jdk8 of IBM Java virtual machine.


* If you’re not familiar with OpenDJ nightly tests, do not try to interpret the results out there. Some of those complex replication tests are unfortunately sensitive to processor speed, thread timing and synchronization.So they tend to fail often on single CPU virtual machines where resources are unknown. They are fully passing on 2 or more CPU machines.

OpenDJ : Visualizing the Replication Topology

My coworker Chris Ridd has spent a little bit of spare time writing a small utility that can parse the output of OpenDJ monitoring information to extract the details of the replication topology. Give the output to some graphical tool and here’s the result (based on one of our biggest customer -anonymized- data) :

ReplTopo

This is a worldwide deployment with many directory services in 4 regions and 8 replication services fully connected. Each directory service is connected to a single replication server, but can failover in matter of seconds, by priority in the same region.

If you want to give it a try on your own replication topology, it’s simple. The tool is open source and part of the OpenDJ utilities that Chris has pushed to GitHub. Just feed it with the output of ldapsearch on cn=monitor.