There is an issue in the start and stop scripts that is preventing OpenDS 2.0 to be installed via Java Web Start on Mac OS X 10.5 with the latest version of the JVM (Update 4 a.k.a 1.6.0_13). I’ve discovered the problem at the same time we were releasing OpenDS 2.0.0 release candidate 4 which was planned to be the last release candidate. So the fix is not the release but has been committed to the trunk.
The issue is that the new JVM does use a larger default minimal heap size and reject any calls with -Xmx if the maximum heap size is smaller than its internal default (around 30MB).
Still OpenDS 2.0 can be installed on Mac OS X and used with the latest JVM, by downloading the Zip file, unzipping it and doing minor edition in the start-ds and stop-ds scripts.
$ unzip ~/Desktop/OpenDS-2.0.0.zip
$ cd OpenDS-2.0.0/bin
In the start-ds and the stop-ds scripts, replace all occurences of the string "-Xms8M -Xmx8M" with "-client"
$ cp start-ds start-ds.orig
$ sed -e "s/-Xms8M -Xmx8M/-client/g" < start-ds.orig > start-ds
$ cp stop-ds stop-ds.orig
$ sed -e "s/-Xms8M -Xmx8M/-client/g" < stop-ds.orig > stop-ds
You can now run the setup program (or launch the QuickSetup application) to install and configure the OpenDS directory server.