In a previous tip, I’ve talked about troubleshooting index problems with OpenDS. Sometimes, one might need to get a better understanding of the OpenDS databases to troubleshoot some data or performance issue.

Here comes the little known OpenDS feature : dbtest.

dbtest is a command line utility that can be used to inspect the content of an OpenDS JE backend (which relies on the Oracle Berkeley Database Java Edition).

The command can be used to list the root container, the entry container, the database containers, get statistics on their content, see the status of index files and possibly dump the content of the database. While dumping the database is hardly useful for most people the tool still has a couple of features of general interest.

It can be used to list all database containers for the userRoot backend, including the index containers, their names and their current entry count.

$ bin/dbtest list-database-containers -b “dc=example,dc=com” -n “userRoot”

Database Name Database Type JE Database Name Entry Count

————————————————————————————————–

dn2id DN2ID dc_example_dc_com_dn2id 102

id2entry ID2Entry dc_example_dc_com_id2entry 102

referral DN2URI dc_example_dc_com_referral 0

id2children Index dc_example_dc_com_id2children 2

id2subtree Index dc_example_dc_com_id2subtree 2

state State dc_example_dc_com_state 19

uniqueMember.equality Index dc_example_dc_com_uniqueMember.equality 0

sn.equality Index dc_example_dc_com_sn.equality 100

sn.substring Index dc_example_dc_com_sn.substring 541

objectClass.equality Index dc_example_dc_com_objectClass.equality 6

entryUUID.equality Index dc_example_dc_com_entryUUID.equality 102

ds-sync-hist.ordering Index dc_example_dc_com_ds-sync-hist.ordering 0

aci.presence Index dc_example_dc_com_aci.presence 0

cn.equality Index dc_example_dc_com_cn.equality 100

cn.substring Index dc_example_dc_com_cn.substring 1137

telephoneNumber.equality Index dc_example_dc_com_telephoneNumber.equality 100

telephoneNumber.substring Index dc_example_dc_com_telephoneNumber.substring 956

givenName.equality Index dc_example_dc_com_givenName.equality 100

givenName.substring Index dc_example_dc_com_givenName.substring 396

uid.equality Index dc_example_dc_com_uid.equality 100

mail.equality Index dc_example_dc_com_mail.equality 100

mail.substring Index dc_example_dc_com_mail.substring 525

member.equality Index dc_example_dc_com_member.equality 0

Total: 23

Or dbtest can be used to retrieve the system and user indexes and their status.

$ bin/dbtest list-index-status -b “dc=example,dc=com” -n “userRoot”

Index Name Index Type JE Database Name Index Status

————————————————————————————————

id2children Index dc_example_dc_com_id2children true

id2subtree Index dc_example_dc_com_id2subtree true

uniqueMember.equality Index dc_example_dc_com_uniqueMember.equality true

sn.equality Index dc_example_dc_com_sn.equality true

sn.substring Index dc_example_dc_com_sn.substring true

objectClass.equality Index dc_example_dc_com_objectClass.equality true

entryUUID.equality Index dc_example_dc_com_entryUUID.equality true

ds-sync-hist.ordering Index dc_example_dc_com_ds-sync-hist.ordering true

aci.presence Index dc_example_dc_com_aci.presence true

cn.equality Index dc_example_dc_com_cn.equality true

cn.substring Index dc_example_dc_com_cn.substring true

telephoneNumber.equality Index dc_example_dc_com_telephoneNumber.equality true

telephoneNumber.substring Index dc_example_dc_com_telephoneNumber.substring true

givenName.equality Index dc_example_dc_com_givenName.equality true

givenName.substring Index dc_example_dc_com_givenName.substring true

uid.equality Index dc_example_dc_com_uid.equality true

mail.equality Index dc_example_dc_com_mail.equality true

mail.substring Index dc_example_dc_com_mail.substring true

member.equality Index dc_example_dc_com_member.equality true

Total: 19

An index status of true means it’s a trusted index, a status of false means the index is no longer trusted and needs

rebuilding.

You can find more details on the dbtest tool on the OpenDS documentation wiki.

Technorati Tags: directory-server, ldap, opends, opensource, tip

The OpenDS development team is very pleased to announce the immediate availability of OpenDS 1.2.0-RC2 which is the second and probably last release candidate for OpenDS 1.2.0. The main goal of the OpenDS 1.2.0 version is to be integrated in the coming release of OpenSolaris.

The purpose of the Release Candidate is to solicit one last round of testing before the final release.

So please test the OpenDS release with your client applications, in your environment or on your favorite platform.

If you do find a bug, please report it with Issue Tracker.

We welcome feedback. Please report you experience with OpenDS on our mailing lists, or on #opends IRC channel on Freenode.

OpenDS 1.2.0-RC2 is built from revision 4920 of the b1.2 branch of our source tree.

The direct link to download the core server is: http://www.opends.org/promoted-builds/1.2.0-RC2/OpenDS-1.2.0-RC2.zip

The direct link to download the DSML gateway is: http://www.opends.org/promoted-builds/1.2.0-RC2/OpenDS-1.2.0-RC2-DSML.war

We have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://www.opends.org/promoted-builds/1.2.0-RC2/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

Detailed information about this build is available at http://www.opends.org/promoted-builds/1.2.0-RC2.

Major changes incorporated since OpenDS 1.2.0-RC1 include:

• Revision 4771 (Issue #3668) – Fix a problem that prevented the Control Panel from correctly displaying a connection handler’s listen addresses.
• Revision 4785 – Fix broken unit tests when attempting to port OpenDS to IBM JVM.
• Revision 4793 (Issue #3676) – Fix an issue in ldapmodify when processing the ;binary option.
• Revision 4806 (Issue #3694) – Fix an error that prevented the ASN.1 package from correctly BER encoding/decoding negative integers.
• Revision 4813 (Issue #3685) – Fix a Swing repainting problem in the control panel.
• Revision 4821 (Issue #3699) – Fix an issue that prevented OpenDS from sending the password Expired Control during a bind operation, if the password had been reset.
• Revision 4828 (Issue #3417) – Allow the import-ldif command to load VLV indexes.
• Revision 4834 (Issue #3710) – Fix a Control Panel error that occurred when creating a new base DN with automatically generated data.
• Revision 4836 (Issue #3705) – Fix a Control Panel problem with the “Save” button in the “Manage Entries” panel.
• Revision 4837 (Issue #3704) – Include the LDIF Connection Handler in the list of connection handlers displayed by the status command.
• Revision 4838 (Issue #3709) – In the Control Panel, change the value of the “Backup Path” field to the instance path rather than the installation path in the Backup/Restore panels.
• Revision 4839 (Issue #3672) – Make it possible to remotely debug the server.
• Revision 4852 (Issue #3511) – Allow password encoding using the schemes 3DES, BLOWFISH, AES, and RC4.
• Revision 4854 (Issue #3579) – Fix an issue that caused the import-ldif countRejects option not to work as expected.
• Revision 4865 (Issue #3683) – Fix an issue with replication conflict resolution for the DELETE operations for entries that have children entries.
• Revision 4866 (Issue #3716) – Fix an issue where start-ds.bat was using the wrong environment variable for passing arguments.
• Revision 4869 (Issue #3707) – Check the validity of all parameters passed to the unconfigure script before starting the unconfiguration.
• Revision 4875 (Issue #3718) – Fix an issue that caused the -A, –typesOnly option to be ignored by the ldapsearch command.
• Revision 4879 (Issue #3723) – Fix an issue that caused the ACI SSF bind rule “!=” operator not to work as expected.
• Revision 4883 (Issue #3725) – Fx an issue with HTML tags incorrectly displayed in the output of the status command.
• Revision 4904 (Issue #3735) – Fix an issue where OpenDS failed to start if the build number contained a version qualifier.
• Revision 4913 (Issue #3750) – Fix an issue in which forcing a password change after an administration reset caused unexpected behavior.
• Revision 4919 (Issue #3621) -Fix an issue that caused the import-ldif, backup and other commands to use the wrong default port.
• Revision 4920 (Issue #3751) – Fix a problem that caused dsreplication initialize-all to fail.

Technorati Tags: directory-server, java, ldap, opends, opensolaris, opensource, Sun

LDAP Directory servers are designed to process search queries at the speed of light (almost).

But sometimes, the search queries issued by a client application are not as fast as expected. This often comes from an indexing misconfiguration or problem, but finding the root cause is not easy. I should say WAS not easy.

The OpenDS LDAP directory server supports a “magic” operational attribute that allows an administrator to get from the server information about the processing of indexes for a specific search query: debugsearchindex.

If the attribute is set in the requested attributes in a search operation, the server will not return all entries as expected, but a single result entry with a fixed distinguished name and a single valued attribute debugsearchindex that contains the information related to the index processing, including the number of candidate entries per filter component, the overall number of candidate, and whether any or all of the search is indexed.

$ bin/ldapsearch -h localhost -p 1389 -D “cn=Directory Manager” -b “dc=example,dc=com” “(&(uid=user*)(mail=joe*))” debugsearchindex

Password for user ‘cn=Directory Manager’: *********

dn: cn=debugsearch

debugsearchindex: filter=(&(uid=user*)[COUNT:100](mail=joe*)[INDEX:mail.equality][COUNT:0])[COUNT:0] final=[COUNT:0]

$ bin/ldapsearch -h localhost -p 1389 -D “cn=Directory Manager” -b “dc=example,dc=com” “objectclass=*” debugsearchindex

Password for user ‘cn=Directory Manager’: *********

dn: cn=debugsearch

debugsearchindex: filter=(objectClass=*)[NOT-INDEXED] scope=wholeSubtree[COUNT:102] final=[COUNT:102]

./ldapsearch -h localhost -p 1389 -D “cn=Directory Manager” -b “dc=example,dc=com” “mail=user.1*” debugsearchindex

Password for user ‘cn=Directory Manager’: *********

dn: cn=debugsearch

debugsearchindex: filter=(mail=user.1*)[INDEX:mail.substring][COUNT:11] scope=wholeSubtree[COUNT:102] final=[COUNT:11]

Technorati Tags: directory-server, ldap, opends, opensource, tip

The OpenDS LDAP directory service has been designed to work well for hosted services. A single directory server can be used to store information for multiple companies either sharing the same database backend, or in separated database backends. The OpenDS tools such as dsconfig or dsreplication focus on the user visible aspect: the Naming Context, also known as the “Base DN”, the starting point of the directory information tree for a given company.

But the backends are not exposed by default, and several users have asked the question “How do I know the backend name for my suffix ?”.

The list-backends command line tool is a handy tool that you can find in the bin directory that can list the backends and suffixes of a Directory Server instance.

Below are a couple of example of use of the list-backends tool:

$ bin/list-backends

Backend ID : Base DN

—————:——————–

adminRoot : cn=admin data

ads-truststore : cn=ads-truststore

backup : cn=backups

config : cn=config

monitor : cn=monitor

schema : cn=schema

tasks : cn=tasks

userRoot : “dc=example,dc=com”

$ bin/list-backends -b “dc=example,dc=com”

The provided DN ‘dc=example,dc=com’ is a base DN for backend ‘userRoot’

You can find more details and examples on the OpenDS documentation wiki’s list-backends reference page.

Technorati Tags: directory-server, ldap, opends, opensource, tip

In the previous tips for OpenDS, the LDAP directory server in Java, I’ve shown the long and exhaustive dsconfig commands. As you may have noticed, there are many properties to specify to reference the server: the hostname, the port number, the authentication DN… Those properties are used by most OpenDS command line tools and repeating them all the time can be tedious.

Fortunately, it is possible to set the default values for the OpenDS command line tools in a property file, stored either in a .opends directory in your home, or in the instance config directory.

Here’s a sample file :

$ cat ~/.opends/tools.properties

port=1389

dsconfig.port=4444

stop-ds.port=4444

hostname=localhost

bindDN=cn=Directory Manager

bindPassword=secret12

If you’re storing passwords in the file, make sure you’re not allowing other users to read the file.

You can find more details on the tools.properties file on OpenDS documentation wiki.

Technorati Tags: directory-server, ldap, opends, opensource, tip

When you install OpenDS LDAP directory server, the server is configured to verify that newly-written or added entries conform to the directory server’s schema (and therefore conform to the LDAP standards).

If you intend to run a reliable service with OpenDS, and provide interoperability between multiple LDAP clients, you should not change this setting. But sometime, developers need to quickly test LDAP with existing data and don’t have the complete definition of the schema, or don’t have time to deal with loading the proper schema. Then, the quick option is to disable schema checking.

This can be done using the dsconfig advanced mode, and the global configuration option check-schema.

dsconfig set-global-configuration-prop \

–set check-schema:false \

–hostname localhost \

–trustAll \

–port 4444 \

–bindDN cn=Directory\ Manager \

–bindPassword ****** \

–no-prompt

There are 2 other properties that can be tuned for a finer grain control of schema checking:

• invalid-attribute-syntax-behavior: controls whether the syntax of the attribute values are checked when adding, modifying entries.
• single-structural-objectclass-behavior: controls how the server should behave if an attempt is made to add or modify an entry with more than one structural objectclass.

You can find more details on schema checking on the OpenDS documentation wiki and more specifically at https://www.opends.org/wiki/page/HowToExtendTheLDAPSchema#section-HowToExtendTheLDAPSchema-ConfiguringSchemaChecking

Technorati Tags: directory-server, ldap, opends, tip

The article published by Gary Williams, QA lead for the OpenDS project, and Marina Sum, technical writer for Sun Developers Network, has been picked by Java.net and promoted on the front page.

Nice work Gary and Marina !

Technorati Tags: developer, directory-server, java, ldap, opends, opensource, QA, quality

When installing OpenDS, you’re asked to define a Base DN (aka Suffix, aka Naming Context) for your directory instance. A default is proposed : dc=example,dc=com.

OpenDS supports multiple suffixes per backend and multiple backends.

The Control Panel allows you to easily add suffixes to the current backend or to a new backend.

You can also do it through the dsconfig command line tool.

$ dsconfig set-backend-prop --backend-name userRoot --add base-dn:dc=MyCompany,dc=com
--hostname localhost --port 4444 --bindDN cn=Directory\ Manager --bindPassword ********
--trustAll --noPropertiesFile --no-prompt

Technorati Tags: directory-server, ldap, opends, tip

 In a previous OpenDS Tip, I talked about dsconfig interactive mode saying that it’s the default mode. When starting with OpenDS and dsconfig, my guess is that the first command tried is dsconfig –help (or -?), and then a more complete command such as dsconfig -h localhost -p 4444 …

But the easiest way to start with dsconfig is just dsconfig with no option… The interactive mode will start querying all parameters to connect to the server before proposing a menu of configuration areas.

$ dsconfig

>>>> Specify OpenDS LDAP connection parameters

Directory server hostname or IP address [dhcp-egnb07-211-104.France.Sun.COM]:

How do you want to trust the server certificate?

1) Automatically trust
2) Use a truststore
3) Manually validate
Enter choice [3]: 1

Directory server administration port number [4444]:

Administrator user bind DN [cn=Directory Manager]:

Password for user 'cn=Directory Manager':

>>>> OpenDS configuration console main menu

What do you want to configure?

1) Access Control Handler      24) Monitor Provider
2) Account Status Notification 25) Network Group Handler
3) Administration Connector    26) Network Group Criteria
4) Alert Handler               27) Network Group Request Filtering Policy
5) Attribute Syntax            28) Network Group Resource Limits
6) Backend                     29) Password Generator
7) Certificate Mapper          30) Password Policy
8) Connection Handler          31) Password Storage Scheme
9) Crypto Manager              32) Password Validator
10) Debug Target               33) Plugin
11) Entry Cache                34) Plugin Root
12) Extended Operation Handler 35) Replication Domain
13) Extension                  36) Replication Server
14) Global Configuration       37) Root DN
15) Group Implementation       38) Root DSE Backend
16) Identity Mapper            39) SASL Mechanism Handler
17) Key Manager Provider       40) Synchronization Provider
18) Local DB Index             41) Trust Manager Provider
19) Local DB VLV Index         42) Virtual Attribute
20) Log Publisher              43) Work Queue
21) Log Retention Policy       44) Workflow
22) Log Rotation Policy        45) Workflow Element
23) Matching Rule
q) quit
Enter choice:

Also, if one of your goal is to be able to script configuration of OpenDS, use the –displayCommand option with the interactive mode: when an configuration setting is done to the server, dsconfig will display the command to use in a script to execute exactly the same configuration setting.

...
 
Enter choice: 14

>>>> Global Configuration management menu
What would you like to do?
1) View and edit the Global Configuration
b) back
q) quit
Enter choice [b]: 1

>>>> Configure the properties of the Global Configuration

Property Value(s)
----------------------------------------------------------------------
1) bind-with-dn-requires-password true
2) default-password-policy Default Password Policy
3) disabled-privilege If no values are defined, then
   the server enforces all privileges.
4) entry-cache-preload false
5) etime-resolution milliseconds
6) idle-time-limit 0 ms
7) lookthrough-limit 5000
8) max-allowed-client-connections 0
9) proxied-authorization-identity-mapper Exact Match
10) reject-unauthenticated-requests false
11) return-bind-error-messages false
12) save-config-on-successful-startup true
13) size-limit 1000
14) smtp-server If no values are defined, then the server cannot
    send email via SMTP.
15) time-limit 1 m
16) workflow-configuration-mode auto
17) writability-mode enabled

?) help
f) finish - apply any changes to the Global Configuration
c) cancel
q) quit
Enter choice [f]: 13

>>>> Configuring the "size-limit" property

Specifies the maximum number of entries that the Directory Server should
return to the client during a search operation.

A value of 0 indicates that no size limit is enforced. Note that this is
the default server-wide limit, but it may be overridden on a per-user
basis using the ds-rlim-size-limit operational attribute.
Syntax: 0 <= INTEGER

Do you want to modify the "size-limit" property?

1) Keep the default value: 1000
2) Change the value
?) help
q) quit

Enter choice [1]: 2

Enter a value for the "size-limit" property [continue]: 2000
Press RETURN to continue

>>>> Configure the properties of the Global Configuration
Property Value(s)
----------------------------------------------------------------------
1) bind-with-dn-requires-password true
2) default-password-policy Default Password Policy
3) disabled-privilege If no values are defined, then
   the server enforces all privileges.
4) entry-cache-preload false
5) etime-resolution milliseconds
6) idle-time-limit 0 ms
7) lookthrough-limit 5000
8) max-allowed-client-connections 0
9) proxied-authorization-identity-mapper Exact Match
10) reject-unauthenticated-requests false
11) return-bind-error-messages false
12) save-config-on-successful-startup true
13) size-limit 2000
14) smtp-server If no values are defined, then the server cannot
    send email via SMTP.
15) time-limit 1 m
16) workflow-configuration-mode auto
17) writability-mode enabled

?) help
f) finish - apply any changes to the Global Configuration
c) cancel
q) quit
Enter choice [f]:

The Global Configuration was modified successfully
The equivalent non-interactive command-line is:

dsconfig set-global-configuration-prop \
 --set size-limit:2000 \
 --hostname dhcp-egnb07-211-104.France.Sun.COM \
 --trustAll \
 --port 4444 \
 --bindDN cn=Directory\ Manager \ 
 --bindPassword ****** \ 
 --no-prompt

Press RETURN to continue

Technorati Tags: directory-server, ldap, opends, tip

OpenDS LDAP directory service comes with a command-line user interface to manage every configuration parameter of the server: dsconfig.

The dsconfig tool can be run in interactive mode (and this is the default) or non interactive mode for use in scripts. Because there are many properties, and most of them would not be changed, some properties are hidden by default.They can still be managed through dsconfig, but with the Advanced mode :

$ dsconfig –advanced

The list-properties command displays the properties sorted by category. You can filter out all properties for a given category with the –category option (-c). And have the advanced properties displayed with the –advanced option.

$ dsconfig list-properties –category global

or with the advanced properties:

$dsconfig list-properties –category global –advanced

Technorati Tags: ldap, opends, tip

This week the spotlights are turned on Gary Williams, the lead test engineer for the OpenDS project. He published a great article with Marina Sum on the topic the OpenDS open source project and Quality Assurance: Perspectives on Quality Assurance for OpenDS.

Gary and the quality assurance team develops tests, ran them on regular basis (over 30.000 unit tests and 2000 functional tests are run daily) and they are key in the reputation and the success of the OpenDS project:

"Our main goal is to deliver a quality product on schedule. As widely acclaimed as OpenDS is for its robust capabilities, we constantly strive for a better-performing product with more user-oriented features."

All tests are part of the project code repository or project documentation. You too can join the project and participate.

You can find more information about OpenDS quality assurance and the "Spirit of Testing" on OpenDS Wiki.

And to paraphrase the conclusion of the article : I feel fortunate and proud to be associated with Gary and his quality team, and look forward to the continued success of OpenDS as a superior global directory service.

Technorati Tags: developer, directory-server, java, opends, opensource, QA, quality