Directory bigots in a lively conversation…

Don BowenSteve ShoaffMy boss Steve Shoaff, Director of Engineering for Directory, and Don Bowen, Distinguished Marketing Director, are discussing the new release of Directory Server Enterprise Edition in a lively and passionate Identity Management Buzz podcast.

 Listen to this episode of the podcast.

You will understand why I really enjoy working with these two bright guys and the rest of the team.

DSEE 6.0 CLI made easier for /bin/bash users

Mark has published several posts on the new Directory Server
Enterprise Edition CLI: dsadm, dsconf for Directory Server and dpadm,
dpconf for Directory Proxy Server [1][2][3][4][5].

Here’s a little trick to facilitate the use of the command line utilities, at least when using  /bin/bash.

The dsconf –help list all available sub-commands, plus a few messages. The first command extract the list of sub-commands and store it in a variable.

ludo:bin > DSC=`dsconf --help | cut -d' ' -f1 | grep -v '^-' | grep -- '-'`

Then we define the list of words to use for completion for the dsconf tool.

ludo:bin > complete -W "`echo $DSC`" dsconf

And we checked that we have a proper completion wordlist for the command. 

ludo:bin > complete -p dsconf
complete -W 'accord-repl-agmt change-repl-dest create-encrypted-attr
create-index create-plugin create-repl-agmt create-repl-priority
create-suffix delete-encrypted-attr delete-index delete-plugin
delete-repl-agmt delete-repl-priority delete-suffix demote-repl
disable-plugin disable-repl disable-repl-agmt enable-plugin enable-repl
enable-repl-agmt get-index-prop get-log-prop get-plugin-prop
get-repl-agmt-prop get-server-prop get-suffix-prop help-properties
init-repl-dest list-encrypted-attrs list-indexes list-plugins
list-repl-agmts list-repl-priorities list-suffixes promote-repl
pwd-compat rotate-log-now set-index-prop set-log-prop set-plugin-prop
set-repl-agmt-prop set-server-prop set-suffix-prop
show-repl-agmt-status show-task-status update-repl-dest-now' dsconf

Use is very simple: type a few characters, hit the [TAB] key, and the command will complete if possible. Hitting [TAB][TAB] will show all available possibilities.

ludo:bin > dsconf create-[TAB][TAB]
create-encrypted-attr  create-plugin          create-repl-priority
create-index           create-repl-agmt       create-suffix
ludo:bin > dsconf create-

The same commands can also work for Directory Proxy Server’s tool: dpconf.

ludo:bin > DPC=`dpconf --help | cut -d' ' -f1 | grep -v '^-' | grep -- '-'`
ludo:bin > complete -W "`echo $DPC`" dpconf
ludo:bin > complete -p dpconf
complete -W 'add-jdbc-attr add-virtual-transformation
attach-jdbc-data-source attach-ldap-data-source
create-connection-handler create-custom-search-size-limit
create-jdbc-data-source create-jdbc-data-source-pool
create-jdbc-data-view create-jdbc-object-class create-jdbc-table
create-join-data-view create-ldap-data-source
create-ldap-data-source-pool create-ldap-data-view
create-ldif-data-view create-request-filtering-policy
create-resource-limits-policy create-search-data-hiding-rule
create-user-mapping delete-connection-handler
delete-custom-search-size-limit delete-jdbc-data-source
delete-jdbc-data-source-pool delete-jdbc-data-view
delete-jdbc-object-class delete-jdbc-table delete-join-data-view
delete-ldap-data-source delete-ldap-data-source-pool
delete-ldap-data-view delete-ldif-data-view
delete-request-filtering-policy delete-resource-limits-policy
delete-search-data-hiding-rule delete-user-mapping
detach-jdbc-data-source detach-ldap-data-source get-access-log-prop
get-attached-ldap-data-source-prop get-connection-handler-prop
get-custom-search-size-limit-prop get-error-log-prop get-jdbc-attr-prop
get-jdbc-data-source-pool-prop get-jdbc-data-source-prop
get-jdbc-data-view-prop get-jdbc-object-class-prop get-jdbc-table-prop
get-join-data-view-prop get-ldap-data-source-pool-prop
get-ldap-data-source-prop get-ldap-data-view-prop
get-ldap-listener-prop get-ldaps-listener-prop get-ldif-data-view-prop
get-request-filtering-policy-prop get-resource-limits-policy-prop
get-search-data-hiding-rule-prop get-server-prop get-user-mapping-prop
get-virtual-aci-prop get-virtual-transformation-prop help-properties
list-attached-jdbc-data-sources list-attached-ldap-data-sources
list-connection-handlers list-custom-search-size-limits list-jdbc-attrs
list-jdbc-data-source-pools list-jdbc-data-sources list-jdbc-data-views
list-jdbc-object-classes list-jdbc-tables list-join-data-views
list-ldap-data-source-pools list-ldap-data-sources list-ldap-data-views
list-ldif-data-views list-request-filtering-policies
list-resource-limits-policies list-search-data-hiding-rules
list-user-mappings list-virtual-transformations remove-jdbc-attr
remove-virtual-transformation rotate-log-now set-access-log-prop
set-attached-ldap-data-source-prop set-connection-handler-prop
set-custom-search-size-limit-prop set-error-log-prop set-jdbc-attr-prop
set-jdbc-data-source-pool-prop set-jdbc-data-source-prop
set-jdbc-data-view-prop set-jdbc-object-class-prop set-jdbc-table-prop
set-join-data-view-prop set-ldap-data-source-pool-prop
set-ldap-data-source-prop set-ldap-data-view-prop
set-ldap-listener-prop set-ldaps-listener-prop set-ldif-data-view-prop
set-request-filtering-policy-prop set-resource-limits-policy-prop
set-search-data-hiding-rule-prop set-server-prop set-user-mapping-prop
set-virtual-aci-prop set-virtual-transformation-prop' dpconf
ludo:bin > dpconf set-ldap[TAB][TAB]
set-ldap-data-source-pool-prop  set-ldap-listener-prop
set-ldap-data-source-prop       set-ldaps-listener-prop
set-ldap-data-view-prop
ludo:bin > dpconf set-ldap 

Add the 4 lines below to your .bashrc to have the completion available in your shells and terminals:

DSC=`dsconf --help | cut -d' ' -f1 | grep -v '^-' | grep -- '-'`
complete -W "`echo $DSC`" dsconf
DPC=`dpconf --help | cut -d' ' -f1 | grep -v '^-' | grep -- '-'`
complete -W "`echo $DPC`" dpconf

Of course, similar commands could be used for dsadm and dpadm as well.

Community events in Paris on Wednesday March 21st

Sun Tech Days are coming to Paris on March 19th, 20th and 21st. As part of this event, there will be a GlassFish Community User Group where we will be presenting OpenDS. Alexis has posted the complete agenda of the meeting as well as other ancillary events.

The same day, still in Paris but at the Sun Customer Briefing Center (Av de Iéna), there is an Identity Management User Group. It seems that pre-registration is very successful, but if you’re a Sun customer and are interested in participating, it is still time.

 See you in Paris in about 2 weeks.

Directory Server 6 HA with Sun Cluster

Directory Server availability is usually obtained with setting up several instances in a  Multi-Master Replication (MMR) topology, but we do also support deployments in a Sun Cluster environment. For the pros and cons of using MMR vs Cluster, you may want to read Neil’s post on the subject.

Before You Start

This cookbook describes how to install
Directory Server as a data service for Sun Cluster 3.1 (or higher) on Solaris 9 or
10 systems, for SPARC, x86, and x64 platforms. You install Directory
Server from native packages by using the Java ES installer.

You must be familiar with Sun Cluster and Directory Server technology in order to find this cookbook useful.
A
detailed How-to guide for setting up a 2 node cluster can be found here.

In following the instructions here, you create one resource group per Directory Server instance.

The example assumes that the machines are in the example.com domain. 

To Prepare Sun Cluster

Start
by preparing the cluster. Directory Service requires an IP address, and
also disk space. Configure the disks in failover mode with affinity set
to on.

Note: You execute cluster commands scrgadm and scswitch only on one node of the cluster.

  1. /etc/hosts and /etc/nsswitch.conf
    • Make sure the logical host name you intend to use is in /etc/hosts (in this example: sun-ldap).
    • Make sure you have "files" before "nis" or "dns" in /etc/nsswitch.conf:
      hosts:      cluster files nis [NOTFOUND=return]
  2. Disks
    • Make
      sure the shared disks that used for the Directory Server instance do
      not have the global option set. Use the "no logging" option, rather
      than the "yes global,logging" in /etc/vfstab:
      /dev/md/sc1/dsk/d50 /dev/md/sc1/rdsk/d50 /clusteredfs/sunds ufs 2 no logging
  3. umount the disks:
    • umount /clusterdisks/sunds
  4. Create the resource group, and the logical hostname:
    • scrgadm -a -g ds-ldap1
    • scrgadm -a -L -g ds-ldap1 -l sun-ldap
  5. Create and configure the disks (HAStoragePlus in failover mode)
    • scrgadm -a -t SUNW.HAStoragePlus
    • scrgadm -a -j disks -g ds-ldap1 -t SUNW.HAStoragePlus -x FilesystemMountPoints=/clusteredfs/sunds -x AffinityOn=TRUE
  6. Enable the resource group
    • scswitch -Z -g ds-ldap1

To Install Directory Server With the Java ES Installer

Install
Directory Server packages on all nodes of the cluster in their default
locations (using the default BASEDIR). Do not use the Java ES installer
to create or to configure a Directory Server instance. Instead, use
dsadm as described in the section "To Create a Directory Server Instance" of this cookbook.

  1. Install Directory Server on all nodes of the cluster:
    • Use JES installer
    • Install all Directory Server software, including the configuration tools.
    • Do not create an instance. Do not configure anything ("configure later" in the Java ES installer).
    • Install Cluster Agents. (This is a specific checkbox in the installer).

To Create a Directory Server Instance

Create
the Directory Server instance on the failover file system. Once
created, manage the instance using Sun Cluster commands. Perform this
procedure on only one node of the cluster.

  1. Make sure the failover filesystem is mounted on the local node:
    • scswitch -z -g ds-ldap1 -h `uname -n`
  2. Create the Directory Server instance:
    • echo secret12 > /tmp/me/password.txt
    • chmod 700 /tmp/me/password.txt
    • dsadm create -w /tmp/me/password.txt -h sun-ldap.example.com /clusteredfs/sunds/myds
      Notes:

    • The logical hostname must be specified when creating the instance,
      otherwise the server will use the node name. This name is used for
      referrals, mostly by the replication feature.
    • If you
      install DS as a non-root user, you need to specify port numbers higher
      than 1024 and make sure you have write permissions in
      /global/sc1/sunds/:

      dsadm create -p 1389 -P 1636 -w /tmp/me/password.txt -h sun-ldap.example.com /clusteredfs/sunds/myds
  3. Make sure the Directory Server instance is properly working:
    • Start the Directory Server instance manually on the node1:
      dsadm start /clusteredfs/sunds/myds
    • Test with an LDAP client to connect to the Directory Server instance:
      ldapsearch -h sun-ldap.example.com -b "" -s base ‘(objectclass=*)’
    • Stop the Directory Server instance:
      dsadm stop /clusteredfs/sunds/myds
    • Switch the Cluster to the other node:
      scswitch -z -g ds-ldap1 -h node2
    • Start the Directory Server instance manually on the other node:
      dsadm start /clusteredfs/sunds/myds
    • Test with an LDAP client:
      ldapsearch -h sun-ldap.example.com -b "" -s base ‘(objectclass=*)’
    • Stop the Directory Server instance:
      dsadm stop /clusteredfs/sunds/myds
  4. Enable the newly created Directory Server instance as a Cluster resource:
    • (as root) scrgadm -a -t SUNW.ds6ldap
    • (as root) dsadm enable-service –type CLUSTER /clusteredfs/sunds/myds ds-ldap1
    • Note: The previous command names the resource with a predefined
      format which is later used to retrieve the DS instance. As a result,
      any attempt to change the resource name will cause the start and stop
      commands to fail. Also, dashes and spaces should be avoided in the
      installation path of the Directory Server instances. These limitations
      may be removed in future versions of DS 6 and its cluster agent.

  5. Work-around bug 6478568 – Missing dependency on disks in dsadm enable-service –type CLUSTER:
    • scrgadm -c -j ds–clusteredfs-sunds-myds -y Resource_dependencies=disks

To Manage a Directory Server Instance

At
this point, only root can stop and start the Directory Server instance,
either with the cluster commands (scswitch -e|-n|-z) on any node of the
cluster, or with the dsadm command:

  • scswitch -e -j ds–clusteredfs-sunds-myds
  • dsadm start /clusteredfs/sunds/myds

DSCC – customizing and troubleshooting

Directory Service Control Center (DSCC) is the new graphical user interface to manage a complete directory service deployment. Below is a screen-shot of the main panel when starting DSCC.

DSCC Screenshot 

DSCC is relying on the Solaris WebConsole, which is available by default on Solaris but has been ported to the other supported platforms (HP-UX, Linux, Windows).

If you want to get a better understanding of the Web Console, want to change its default configuration or need to troubleshoot it, please refer to this document: http://docs.sun.com/app/docs/doc/817-1985/6mhm8o5ke?a=view.

It’s time to upgrade your Directory Service…

Sun Java System Directory Server Enterprise Edition 6.0 (DSEE) was released today along with Sun Java Enterprise System 5.

They are available for download immediately.

The DSEE 6.0 Evaluation Guide (one of the new guide out of the complete documentation set) contains a quick overview of the new features, help on how to get started and much more.

You may also want to check Mark, Jonathan, Neil‘s blogs in the coming days and weeks for more information about Directory Server Enterprise Edition 6.0.

 Go, get our product, play with it and have fun !

 [Update on March 6th] For smaller downloads than the complete Java ES 5,  DSEE 6.0 and the Identity Management Suite can be downloaded from http://www.sun.com/software/swportfolio/get.jsp. Select the Identity Management Suite, and click the Get Downloads and Media button at the bottom.

Directory Server Enterprise Edition 6.0 docs are live…

As Mark pointed out yesterday, the Directory Server Enterprise Edition 6.0 documentation set went live at http://docs.sun.com/coll/1224.1.

The product should be available for download very soon.

LDAP C-SDK: Merge back to Mozilla.org is done…

My colleague Anton has finished contributing back to the Mozilla.org LDAP C-SDK all of the changes, enhancements, bug fixes that we’ve been accumulated for a few years. This is available in the trunk, starting with version 6.0.2.

The LDAP C-SDK includes the C ldap libraries as well as the tools such as ldapsearch, ldapmodify and newly added ldappasswd.

The merge is a very good thing for customers and developers since there is now a single source for tools and libraries that will be delivered in Sun’s products, Solaris, Fedora and Red-Hat Linux.

Builds are not available yet, but Sun will start contributing, at least Solaris builds, since they will be the ones that we will be using from now on for Sun Java Enterprise Systems and its flagship product: Sun Java System Directory Server Enterprise Edition (slight bias 😉 ). And an up-to-date and refreshed documentation of the LDAP C-SDK and tools is on its way as well.

We owe a big thank to everyone involved and more specifically Mark,
Rich, Noriko and  Nathan, most of which we’ve worked with during the
iPlanet days (the Sun | Netscape Alliance). Despite the fact that we’ve chosen different directions, it’s still  a pleasure to work with you guys.

 

 

LDAP programming: JNDI tutorial revisited

For those who’ve been built Java based applications for LDAP using Java Naming and Directory Interface(tm) (JNDI), the JNDI tutorial has surely been a great help, but long and hard to browse through.

Jaya has just published a new JNDI Trail, shorter and updated to cover the JDK5.0 and JDK6.0 features: a reference to keep for any LDAP / Java developer.

The trail is also mentioned on the Java Tutorials Weblog.

Super Hero ?

I am sure that for my 3 daughters I am THE super-hero. But I don’t know if they would reach the same conclusion as this quiz:

You are Spider-Man

Spider-Man
75%
Robin
65%
The Flash
60%
Supergirl
55%
Superman
55%
Iron Man
45%
Hulk
40%
Green Lantern
35%
Wonder Woman
20%
Catwoman
20%
Batman
20%
You are intelligent, witty,
a bit geeky and have great
power and responsibility.


Click here to take the Superhero Personality Quiz

Joyeux Noël et Bonnes Fêtes…

It’s time to take a well deserved break with my wife and daughters, visit the family and relatives.

I’ll be back early January…

Meanwhile, I wish you all a Merry Christmas (or in French: Joyeux Noël).

 

Directory Services in the Telco world

Last week I was invited to a meeting with one of our customers,
a wireless telecom operator happily  user of Sun Directory Server 5.2 (patch3) with a few tens of million entries.

With the convergence of voice and data, the telcos are looking
for ways to reduce the number of databases they have and consolidate the
data in a single repository such as LDAP-based directory services.
The discussion went on the subject of the data models, the differences
between the LDAP model and the relational model, drifting to which model would be the
most appropriate in consideration with the Generic User Profile recommendation from the IMS specifications. Clearly the discussion was reaching the limits of my expertise (while
I’m quite confident in the LDAP area, IMS is not something that I’ve
followed), but it was very informative.

The one thing that I really found interesting in this discussion: at no
time, the consideration of performances was mentioned. It seemed obvious
for all parties that LDAP directory services (and probably more
specifically our Directory Server) do have the capability of keeping
with the high throughput and low response time requirements of the
network equipments.

And in fact, they really do. We will have some evidence of this with Directory
Server Enterprise Edition
6.0 very soon.

OpenDS crossing ELBE

No, OpenDS is not moving to Central Europe (where the Elbe is a major waterway).

Matthew, developer on OpenDS and located in an office near mine, showed me how ELBE, an Eclipse plugin for Browsing and Editing LDAP directories, could be with OpenDS and more specifically the instance running on his laptop :

Snapshot ELBE OpenDS 

ELBE OpenDS Schema 

ELBE is really a cool and well designed tool… I personally just wished it runs in my preferred IDE: NetBeans.

 

Is this the biggest Directory Server in production ?

Another great Sun customer story has just been published on http://www.sun.com.

This time it’s about Sina, one of the largest Web portals and a leading online media and value-added information service provider in China, who redeployed its Sun Java System Directory Server on 12 Sun Fire T1000 servers, powered by coolthreads.

But with over 230 million users in Sun Java System Directory Server, I believe that this is the largest Directory Server in production.

I’m amazed with what our customers are doing with our product, and I’m sure that this "record" (if it is one) will not stand long with Directory Server Enterprise Edition 6.0 coming soon, enabling new kinds of highly scalable and manageable directory services.

Addendum on Dec 20, 2006:

It seems that the current number of entries in the Directory is more around 120 than 230 as the article suggested. But according to one of the engineers involved in the project, the plan is to move to 600 millions soon. I’m still waiting for the deployment details: size of data, partitioning and performance numbers. May be early next year 😉

OpenSSO using OpenDS…

My colleague Indira has just posted a guide on how to use OpenDS as the backend for OpenSSO.

Nice piece of work, Indira.

An other level of integration would be to completely embed OpenDS in OpenSSO to provide a single server with highly efficient storage. For single server deployment, it might be easier to manage.