The Agenda for the 2nd International LDAP Conference (LDAPCon 2009).

The agenda for the 2nd International LDAP Conference has been published, and the conference really looks very attractive to me (sure I’ll be presenting and thus I’m a little bit biased Angelic). I can’t wait to be in Portland OR, meet with all the LDAP experts from the various open source projects or directory services vendors and have those lively discussions.

I hope to see you there.

Technorati Tags: , , ,

Updated schema-convert.py script for OpenDS.

Opends Logo TagBack in March, in the series of OpenDS tips, I had written an article and a script to help converting schema files for OpenLDAP to a format suitable for OpenDS.

I received multiple suggestions for the scripts as well as sample schema files that would not load in OpenDS after being converted. Most of the time, it was due to the lack of respect of the RFC syntaxes.

Recently, I found the time to update the script and produce better conversion. You can get the python script here. The March article has been refreshed with the new script as well.

Technorati Tags: , , , , ,

OpenDS turns 3 today…

OpenDS open source project is 3 years oldAnother year has passed and we already end of July. Today is the anniversary day for the OpenDS project which is turning 3 this year.

As usual, this is also time to look back in the mirror and consider what we’ve achieved.

A little more than 10 days ago, we announced the availability of OpenDS 2.0, the new and stable release of our LDAPv3 directory server. OpenDS 2.0 came just about one year after OpenDS 1.0 and 6 months after OpenDS 1.2.

You can read about OpenDS 2.0 features in the Release Notes, but also in the various articles that have relayed our own announcement such as:

Sun OpenDS Standard Edition 2.0 CD
Yesterday, Sun publicly announced the general availability of Sun OpenDS Standard Edition 2.0, a Sun supported version of the OpenDS project, as well as the release of OpenSSO Express Build 8 (due in a couple of weeks).

Sun OpenDS Standard Edition 2.0 has the same features as OpenDS 2.0. Differences are in the branding, the license, the documentation that is available from docs.sun.com in HTML and PDF and of course the support services offered by Sun.

Mark Craig has already posted an illustrated article describing how easy it was to install Sun OpenDS Standard Edition 2.0 on Windows XP.

OpenSSO Express builds are supported snapshots of OpenSSO development. As Pat Patterson, Community Manager for OpenSSO and covering all Identity Products at Sun, detailed on his blog, OpenSSO Express Build 8 includes a new Mobile One Time Password Feature, the Fedlet for .Net and a new task flow enabling single sign-on to Salesforce.com.

As OpenDS is getting mature, we’re seeing public endorsement and use of it. In the last couple of weeks, we had 2 success stories including the use of OpenDS :

Finaly within a year, the OpenDS Community has more than doubled, in term of members in the community, but as well in the number of active contributors and participants in the #opends IRC channel, and in term of unique visitors on the www.OpenDS.org.

OpenDS.org Monthly visits

I’m proud of what we’ve accomplished in 3 years and even more of the past year. We still have a lot of ideas and customers requirements to build in the OpenDS project. Overall we know where we want to go and we hope our new executives will agree that it’s a nice and viable path to follow…

Technorati Tags: , , , , , , , ,

OpenDS 2.0 on Mac OS X with the latest JVM…

Opends2 PictoMacOSX 10.5.7There is an issue in the start and stop scripts that is preventing OpenDS 2.0 to be installed via Java Web Start on Mac OS X 10.5 with the latest version of the JVM (Update 4 a.k.a 1.6.0_13). I’ve discovered the problem at the same time we were releasing OpenDS 2.0.0 release candidate 4 which was planned to be the last release candidate. So the fix is not the release but has been committed to the trunk.

The issue is that the new JVM does use a larger default minimal heap size and reject any calls with -Xmx if the maximum heap size is smaller than its internal default (around 30MB).

Still OpenDS 2.0 can be installed on Mac OS X and used with the latest JVM, by downloading the Zip file, unzipping it and doing minor edition in the start-ds and stop-ds scripts.

$ unzip ~/Desktop/OpenDS-2.0.0.zip

Archive: /Users/ludo/Desktop/OpenDS-2.0.0.zip

creating: OpenDS-2.0.0/



inflating: OpenDS-2.0.0/upgrade

$ cd OpenDS-2.0.0/bin

In the start-ds and the stop-ds scripts, replace all occurences of the string "-Xms8M -Xmx8M" with "-client"

$ cp start-ds start-ds.orig

$ sed -e "s/-Xms8M -Xmx8M/-client/g" < start-ds.orig > start-ds

$ cp stop-ds stop-ds.orig

$ sed -e "s/-Xms8M -Xmx8M/-client/g" < stop-ds.orig > stop-ds

OpenDS QuickSetup App IconYou can now run the setup program (or launch the QuickSetup application) to install and configure the OpenDS directory server.

Technorati Tags: , , , , , ,

Configuring Replication Groups: A small but important new feature of OpenDS 2.0

Opends2.0

I’m mostly done with the series of post concerning the new features of the latest release of OpenDS, the opensource LDAPv3 directory service. Yesterday, Mathieu, the developer behind Assured Replication, reminded me of a small but important new feature of OpenDS, in the area of replication: the ability to configure Replication Groups.

A replication group is a simple way to relate replicated OpenDS directory servers together. It’s useful when there are more than 2 replicated servers, when the replicated servers are within different data-centers or to distinguished primary servers from secondary servers.

Replications groups are identified by a group ID which is a unique number assigned to a replication domain on a directory server and to replication servers. Group IDs determine how a directory server domain connects to an available replication server. From the list of configured replication servers, a directory server first tries to connect to a replication server that has the same group ID as that of the directory server. If no replication server with a compatible group ID is available, the directory server connects to a replication server with a different group ID.

In practice, it allows to prioritize how the replication traffic is done between the servers. In the case of multiple data-centers, it’s preferable that all directory servers in a data-center are connected to replication servers in the same data-center. Only in the case of absence of a local replication server, a directory server will connect to a remote replication server.



Note that when configuring replication with OpenDS 2.0 and the
dsreplication utility, both the replication server and the directory server are configured in the same process and thus the same host. It will be very rare if the replication server is not working for its directory server.



The figure below is an illustration of 2 Replication Groups, one for each data center.

OpenDS 2.0 Replication Groups with multiple data-centers

Now to configure a replication group ?

A replication group is configured on each directory server and replication server that should be part of the same group.

On the directory server, the replication group is configured per replication domain (i.e. per replicated suffix).

First identify the replication domain

$ bin/dsconfig -D “cn=directory manager” -j /tmp/passwdfile -n -s list-replication-domains –provider-name “Multimaster Synchronization”

cn=admin data (domain 29167)

cn=schema (domain 9674)

dc=example,dc=com (domain 14741)

Then set the group ID for the domain

$ bin/dsconfig -D “cn=directory manager” -j /tmp/passwdfile -n set-replication-domain-prop –provider-name “Multimaster Synchronization” –domain-name “dc=example,dc=com (domain 14741)” –advanced –set group-id:5

For the replication server

$ bin/dsconfig -D “cn=directory manager” -j /tmp/passwdfile -n set-replication-server-prop –provider-name “Multimaster Synchronization” –advanced –set group-id:5

Repeat this to the other directory servers and replication servers that should be part of the same group.

Note that there is a group by default with the group ID 1.

Configuring replication groups have some impact when using Assured Replication, since Assured Replication only works within a single group. So groups can be used to limit the impact of network latency when using Assured Replication, or to constrain the changes to be more consistent in a single data-center.

You can find more information about replication groups in the Replication Architecture reference manual and in the Replication section of the Administration Guide.

Technorati Tags: , , , , ,

Assured Replication: A New Feature of OpenDS 2.0

OpenDS 2.0 has just been released and there are several new and exciting features in it.

To me, the biggest innovation in this release is "Assured Replication", an extension to the loose consistency multi-master replication feature that brings tighter consistency of data between replica. "Assured Replication" is not to be taken for a full synchronous and transactional replication mechanism. A change is not transactionally applied to a set of or all replicas of a topology. With "Assured Replication", the response to an LDAP modification is delayed until the change has been received or applied by other servers, in a best effort mode. It provides a greater assurance that a change is not lost even if the server receiving it crashes.

Opends Assured Replication with Safe Data level 2

Assured Replication can function in 2 modes :

  • Safe Data Mode: an update must be propagated to a defined number of Replication Servers before returning a response to the client. So if the server or the replication server is stopped, the data is still available to all other replicas.
  • Safe Read Mode: an update must be propagated to all directory servers in the domain before the client is returned a response for the update.

Of course, for both modes, it’s possible to configure a timeout interval to prevent LDAP clients to be waiting indefinitely if some servers are not available.

Configuring Assured Replication is pretty straightforward but cannot be done when setting up replication itself. So the first step is to configure Multi-Master Replication for a domain with dsreplication.

$ bin/dsreplication enable –host1 localhost –port1 5444 –bindDN1 ‘cn=directory manager’ –bindPassword1 secret12 –replicationPort1 8989 –host2 localhost –port2 6444 –bindDN2 ‘cn=directory manager’ –bindPassword2 secret12 –replicationPort2 8990 –adminUID admin –adminPassword secret12 –baseDN "dc=example,dc=com" -X -n

Establishing connections ….. Done.

Checking Registration information ….. Done.

Configuring Replication port on server localhost:5444 ….. Done.

Configuring Replication port on server localhost:6444 ….. Done.

Updating replication configuration for baseDN dc=example,dc=com on server localhost:5444 ….. Done.

Updating replication configuration for baseDN dc=example,dc=com on server localhost:6444 ….. Done.

Updating Registration configuration on server localhost:5444 ….. Done.

Updating Registration configuration on server localhost:6444 ….. Done.

Updating replication configuration for baseDN cn=schema on server localhost:5444 ….. Done.

Updating replication configuration for baseDN cn=schema on server localhost:6444 ….. Done.

Initializing Registration information on server localhost:6444 with the contents of server localhost:5444 ….. Done.

Initializing schema on server localhost:6444 with the contents of server localhost:5444 ….. Done.

Replication has been successfully enabled. Note that for replication to work you must initialize the contents of the base DN’s that are being replicated (use dsreplication initialize to do so).

$ bin/dsreplication initialize –baseDN "dc=example,dc=com" –adminUID admin –adminPassword secret12 –hostSource localhost –portSource 5444 –hostDestination localhost –portDestination 6444 -X -n

Initializing base DN dc=example,dc=com with the contents from localhost:5444:

23 entries processed (100 % complete).

Base DN initialized successfully.

See

/var/folders/SH/SHFsRjymHtqiZ4GxPNZERU++Fwk/-Tmp-/opends-replication-737929812662715818.log

for a detailed log of this operation.

$ bin/dsreplication status -h localhost -p 5444 –adminUID admin –adminPassword secret12 -X

dc=example,dc=com – Replication Enabled

=======================================

Server : Entries : M.C. (1) : A.O.M.C. (2) : Port (3) : Security (4)

—————:———:———-:————–:———-:————-

localhost:5444 : 23 : 0 : N/A : 8989 : Disabled

localhost:6444 : 23 : 0 : N/A : 8990 : Disabled

Now that replication is setup, we can enable the Assured Replication mode, using the dsconfig utility. For this, on each of the OpenDS direcotry servers, we first need to retrieve the full name of the replication domain.

$ bin/dsconfig -D cn=directory\ manager -w secret12 -n -s list-replication-domains –provider-name "Multimaster Synchronization"

cn=admin data (domain 29167)
cn=schema (domain 9674)
dc=example,dc=com (domain 14741)

$ bin/dsconfig -D cn=directory\ manager -w secret12 -n set-replication-domain-prop –provider-name "Multimaster Synchronization" –domain-name "dc=example,dc=com (domain 14741)" –advanced –set assured-type:safe-data –set assured-sd-level:2

Note that the Replication Domain has a different value on each server, so you have to repeat these 2 commands on each instance.

Setting the assured level for Safe Data to 2 means that the server will make sure the data has been received by at least 2 replication services before returning to the LDAP client the response to the update request.

From a client point of view, there should be no difference, except that the server might take a little longer to return the response to an update request. In our measures, we found that the response time increased by 25% for Safe Data Level 2, which seems a lot, but honestly, when the response time is in the order of 2ms, it’s hard to notice !

You can find more information about Assured Replication on OpenDS 2.0 documentation wiki, both in the overview of OpenDS Replication Architecture and the Replication Administration Guide, and more specifically Assured Replication Administration Guide

Technorati Tags: , , , , ,