Open Provisioning ToolKit phoenix moment

OpenPTKI’m sitting in training this week with our Solution Architects team and was talking to my long time colleague Scott Fehrman about a customer I recently met and a mention of the Open Provisioning ToolKit (OpenPTK) in one of the slides. OpenPTK is an open source project that Scott, Terry Sigle and Derrick Harcey founded at Sun Microsystems some years ago.

As we’re talking Scott realized that the website that hosted OpenPTK source code, issues and downloads (java.net) is gone. As he had a copy of the latest version, he put it back online on his github account.

If anyone is using OpenPTK and would like to get the code, or even better work on it, it has raised from the hashes and is now publicly available:

https://github.com/sfehrman/openptk

Managing OpenDJ with REST

OpenDJ, the open source LDAP Directory Server, was the first to propose a native HTTP REST / JSON access to the data.

In the next major release, OpenDJ will be providing many enhancements to the REST interface, that I will describe in a series of posts. To start with, let’s talk about the new administrative interfaces added to manage the OpenDJ server.

When the HTTP access is enabled, OpenDJ creates by default 2 administrative endpoints: /admin/config and /admin/monitor.

/admin/config provides a read-write access to the configuration, with the same view and hierarchy of objects as the LDAP access. All of the operations that are possible with the dsconfig command, can be done over LDAP, and now REST.  As a matter of fact, the /admin/config API is automatically generated from the same XML description files that are used to generate the LDAP view and the dsconfig command line utilities. This means that any extension, plugin added to the server will also be exposed via REST without additional code.

screen-shot-2016-10-25-at-15-03-54

Above is an example of query of the /admin/config endpoint, querying for all  backends , done as a user who has the privilege to read the configuration. A similar query done with a user that doesn’t have the config-read privilege does fail as below:

$ curl -s -u user.2 http://localhost:8080/admin/config/backends/userRoot
Enter host password for user 'user.2': 
{
 "message" : "Insufficient Access Rights: You do not
have sufficient privileges to perform search operations
in the Directory Server configuration",
 "code" : 403,
 "reason" : "Forbidden"
}

/admin/monitor provides a read-only view on all of the OpenDJ monitoring information that was already accessible via LDAP under the "cn=Monitor" naming context, and JMX.

$ curl -s -u user.0 http://localhost:8080/admin/monitor/
Enter host password for user 'user.0':
{
 "_id" : "monitor",
 "upTime" : "0 days 2 hours 49 minutes 54 seconds",
 "currentConnections" : "1",
 "totalConnections" : "32",
 "currentTime" : "20161024103215Z",
 "startTime" : "20161024074220Z",
 "productName" : "OpenDJ Server",
 "_rev" : "00000000644a67b2",
 "maxConnections" : "3"
}

The /admin REST endpoints can be protected with different authorization mechanisms, from HTTP basic to OAuth2. And the whole endpoint can be disabled as well if needed using dsconfig.

These administrative REST endpoints can be tested with the OpenDJ nightly builds. They are also available to ForgeRock customers as part of our latest update of the ForgeRock Identity Platform.

Storing JSON objects in LDAP attributes…

jsonUntil recently, the only way to store a JSON object to an LDAP directory server, was to store it as string (either a Directory String i.e a sequence of UTF-8 characters, or an Octet String i.e. a blob of octets).

But now, in OpenDJ, the Open source LDAP Directory services in Java, there is now support for new syntaxes : one for JSON objects and one for JSON Query. Associated with the JSON query, a couple of matching rules, that can be easily customised and extended, have been defined.

To use the syntax and matching rules, you should first extend the LDAP schema with one or more new attributes, and use these attributes in object classes. For example :

dn: cn=schema
objectClass: top
objectClass: ldapSubentry
objectClass: subschema
attributeTypes: ( 1.3.6.1.4.1.36733.2.1.1.999 NAME 'json'
SYNTAX 1.3.6.1.4.1.36733.2.1.3.1 EQUALITY caseIgnoreJsonQueryMatch SINGLE-VALUE )
objectClasses: (1.3.6.1.4.1.36733.2.1.2.999 NAME 'jsonObject'
SUP top MUST (cn $ json ) )

Just copy the LDIF above into config/schema/95-json.ldif, and restart the OpenDJ server. Make sure you use your own OIDs when defining schema elements. The ones above are samples and should not be used in production.

Then, you can add entries in the OpenDJ directory server like this:

$ ldapmodify -a -D cn=directory\ manager -w secret12 -h localhost -p 1389

dn: cn=bjensen,ou=people,dc=example,dc=com
objectClass: top
objectClass: jsonObject
cn: bjensen
json: { "_id":"bjensen", "_rev":"123", "name": { "first": "Babs", "surname": "Jensen" }, "age": 25, "roles": [ "sales", "admin" ] }

dn: cn=scarter,ou=people,dc=example,dc=com
objectClass: top
objectClass: jsonObject
cn: scarter
json: { "_id":"scarter", "_rev":"456", "name": { "first": "Sam", "surname": "Carter" }, "age": 48, "roles": [ "manager", "eng" ] }

The very nice thing about the JSON syntax and matching rules, is that OpenDJ understands how the values of the json attribute are structured, and it becomes possible to make specific queries, using the JSON Query syntax.

Let’s search for all jsonObjects that have a json value with a specific _id :

$ ldapsearch -D cn=directory\ manager -w secret12 -h localhost -p 1389 -b "dc=example,dc=com" -s sub "(json=_id eq 'scarter')"

dn: cn=scarter,ou=people,dc=example,dc=com
objectClass: top
objectClass: jsonObject
json: { "_id":"scarter", "_rev":"456", "name": { "first": "Sam", "surname": "Carter" }, "age": 48, "roles": [ "manager", "eng" ] }
cn: scarter

We can run more complex queries, still using the JSON Query Syntax:

$ ldapsearch -D cn=directory\ manager -w secret12 -h localhost -p 1389 -b "dc=example,dc=com" -s sub "(json=name/first sw 'b' and age lt 30)"

dn: cn=bjensen,ou=people,dc=example,dc=com
objectClass: top
objectClass: jsonObject
json: { "_id":"bjensen", "_rev":"123", "name": { "first": "Babs", "surname": "Jensen" }, "age": 25, "roles": [ "sales", "admin" ] }
cn: bjensen

For a complete description of the query  filter expressions, please refer to ForgeRock Common  REST (CREST) Query Filter documentation.

The JSON matching rule supports indexing which can be enabled using dsconfig against the appropriate attribute index. By default all JSON fields of the attribute are indexed.

In a followup post, I will give more advanced configuration of the JSON Syntax, detail how to customise the matching rule to index only specific JSON fields, and will outline some best practices with the JSON syntax and attributes.

OpenDJ Contact Manager for Android

With OpenDJ 2.6.0, we’ve introduced a new way to access your directory data, using HTTP, REST and JSon. The REST to LDAP service, available either embedded in the OpenDJ server or as a standalone web application, is designed to facilitate the work of application developers. And to demonstrate the interest and the ease of use of that service, we’ve built a sample application for Android : the OpenDJ Contact Manager

OpenDJ Contact Manager Android AppAbout screen of the OpenDJ Contact Manager Android App

The OpenDJ Contact Manager is an open source Android application that was built by Violette, one of the ForgeRock engineer working in the OpenDJ team. You can get the source code from the SVN repository : https://svn.forgerock.org/commons/mobile/contact-manager/trunk. Mark wrote some quite complete documentation for the project, with details on how to get and build the application. He published it at http://commons.forgerock.org/mobile/contact-manager/.

The whole application is just about 4000 lines of code, and most of it is dealing with the display itself. But you can find code that deals with asynchronous calls to the OpenDJ rest interface, with paging through results, and parsing the resulting JSON stream to populate the Contacts, including photos. Et voila :

OpenDJ Contact Manager displaying a Contact

The application is just a sample but it clearly is usable in its current form and will allow once a contact was retrieved from the OpenDJ directory, to add it to the Contacts standard application, call the person, locate its address on maps, send the person an email, navigate through the management chain…

In future versions, we are planning to add support for OAuth 2.0, removing the need to store credentials in the application settings.

As it’s open source, feel free to play with it, hack and contribute back your changes.

OpenDJ 2.6.0 is now available

OpenDJ-300x100I am really happy to announce the general availability of OpenDJ 2.6.0, a major update of ForgeRock  directory service product, built from the tag 2.6.0 (revision 9086 in our SVN repository).

OpenDJ 2.6.0 brings a lot of added value, including :

– A REST to LDAP service, allowing an easy access to directory data using HTTP/JSON. The service can be run either embedded in the server or as a standalone web application.

– A new upgrade process to ease transition from OpenDJ 2.4.5 or newer to 2.6.

– New Linux native packages (RPM and Debian) to facilitate the automatic deployment of OpenDJ in the private and public cloud.

– OpenDJ can be configured to delegate authentication to a Microsoft Active Directory service, providing tighter integration with Microsoft environment without the burden of synchronizing passwords.

– An optional extension to remove specific attributes from updates, making it more flexible and easier to deal with legacy applications and migration tasks.

– A way to synchronize SAMBA password attributes with the user’s password.

– Some improvements on the integrity of references, that is now enforced at creation or on update.

– More flexible and efficient audit logs.

– A Java based LDAP software development kit.

– An official stable documentation.

For the complete list of new features, enhancements and fixed defects, please read the release notes.

The binaries can be downloaded from ForgeRock Downloads.

Over the course of the development of OpenDJ, we’ve received many contributions, in form of code, issues raised in our JIRA, documentation… We address our deepest thanks to all the contributors and developers :

Aiman Tahboub, Alan Evans, Arturo V Sanchez, Auke Schrijnen, Bernhard Thalmayr, Brent Palmer, Bruno Vernay, Chris Dowey, Chris Ridd, Christophe Sovant, Dan Gardner, Danny Turner, Darin Perusich, Donal Duane, Elliot Kendall, Eswar Moorthy, Fred Voss, Gael Allioux, Gary Williams, German Parente, Göran Odmyr, Ian McGlothlin, Jamie Nelson, Jean-Noël Rouvignac, Jeff Blaine, Jeffrey Crawford, Jens Elkner, Lana Frost, Laurent Bristiel, Ludovic Poitou, Manuel Gaupp, Manuel Schallar, Mark Craig, Mark Gibson, Marko Harjula, Martin Sperle, Matthew Stevenson, Matthew Swift, Miroslav Fadrhonc, Mitch Silverstein, Nemanja Lukić, Nicholas Sushkin , Nikolay Belaevski, Per-Olov Sjoholm, Peter Major, Rauli Ikonen, Sachiko Wallace, Slavomir Katuscak, Tomas Forsman, Vanessa Richie, Violette Roche, Willi Burmeister

Happy 4th of July everyone !