Directory Server Enterprise Edition 6.x is built on SleepyCat Berkeley DB for the storage of the LDAP entries and indexes…
For many years, customers have requested that we provide a way to shrink the database files, reclaiming unused pages. Well, the version of the SleepyCat BDB that we are using with DS 6.1 now has a public API that offer this capability and we’ve added the feature in DSEE 6.1. There is now a new subcommand for dsadm the offline DS management CLI: repack.
The usage is the following:
ludo:dsee63 > ds6/bin/dsadm repack --help Usage: dsadm repack [ -b ] INSTANCE_PATH SUFFIX_DN [SUFFIX_DN ...] Repacks existing suffix The accepted value for OPTIONS is: -b, --backend Enables to specify backend name instead of SUFFIX_DN For global options, use dsadm --help. INSTANCE_PATH Path of the Directory Server instance SUFFIX_DN Suffix DN (Distinguished Name) to repack For more information, see dsadm(1M).
It operates on a Suffix and all DB files for this suffix are compacted. This includes the main data file (id2entry), all index files but also the replication changelog file if the suffix is Replicated and configured as a Master or Hub Replica. The compaction will process up to 8 files in parallel, each in its own thread.
Because database compaction is very IO intensive and requires exclusive access to the database, the Directory Server must be stopped to run this.
The time to do the compaction varies a lot depending on the overall size of the database, the number of updates done on the data since the creation of the suffix or last compaction, and mostly the performance of the disk subsystem. Or course, the larger the DB and the more changes, the longer it’ll take but the more it’ll regain disk space. In our experience with a database of approximately 10 million entries and many changes done since creation, the compaction process took approximately a couple of hours (on a v20 z).
Definitely, database compaction is not something that you want to run in your weekly maintenance routine. The downtime of Directory Server may be too long. But if the available disk’s space is getting close to the low watermark, it is worth stopping the server and reclaiming some of the space, before thinking of expanding the disk partition.
Below is an example of the dc=example,dc=com DB files before and after compaction, and the commands used to do the compaction: stop the server, compact, start the server.
ludo:dsee63 > ll instances/ds1/db/example/ total 3640 drwx------ 2 lpoitou icnc 1536 Jul 3 10:45 . drwxr-xr-x 3 lpoitou icnc 512 Jul 9 17:39 .. -rw------- 1 lpoitou icnc 38 Jul 3 10:45 DBVERSION -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_aci.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 17:39 example_ancestorid.db3 -rw------- 1 lpoitou icnc 147456 Jul 9 17:39 example_cn.db3 -rw------- 1 lpoitou icnc 122880 Jul 9 17:39 example_entrydn.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_givenName.db3 -rw------- 1 lpoitou icnc 835584 Jul 9 17:39 example_id2entry.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_mail.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_mailAlternateAddress.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_mailHost.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_member.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_nsCalXItemId.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_nscpEntryDN.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_nsds5ReplConflict.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_nsLIProfileName.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_nsRoleDN.db3 -rw------- 1 lpoitou icnc 131072 Jul 9 17:39 example_nsuniqueid.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_nswcalCALID.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_numsubordinates.db3 -rw------- 1 lpoitou icnc 32768 Jul 9 17:39 example_objectclass.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_owner.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 17:39 example_parentid.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_pipstatus.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_pipuid.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_pwdaccountlockedtime.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_pwdfailuretime.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_pwdgraceusetime.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_seeAlso.db3 -rw------- 1 lpoitou icnc 139264 Jul 9 17:39 example_sn.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_telephoneNumber.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_uid.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_uniquemember.db3 ludo:dsee63 > ds6/bin/dsadm stop /local/demo/dsee63/instances/ds1 Server stopped ludo:dsee63 > ds6/bin/dsadm repack /local/demo/dsee63/instances/ds1 dc=example,dc=com [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example' [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', LDAP entries, size 827392 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index aci, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index ancestorid, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index ancestorid finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index aci finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index cn, size 139264 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', LDAP entries finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index cn finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index mailHost, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index mailHost finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index member, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index member finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nsCalXItemId, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nsCalXItemId finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nscpEntryDN, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nscpEntryDN finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nsds5ReplConflict, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nsds5ReplConflict finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nsLIProfileName, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nsLIProfileName finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nsRoleDN, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nsRoleDN finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nsuniqueid, size 122880 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nswcalCALID, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nswcalCALID finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index nsuniqueid finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index entrydn, size 122880 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index numsubordinates, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index numsubordinates finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index entrydn finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index givenName, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index givenName finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index parentid, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index parentid finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index pipstatus, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index pipstatus finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index pipuid, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index pipuid finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index pwdaccountlockedtime, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index pwdaccountlockedtime finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index pwdfailuretime, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index pwdfailuretime finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index pwdgraceusetime, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index pwdgraceusetime finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index seeAlso, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index seeAlso finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index sn, size 131072 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index uid, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index uid finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index sn finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index owner, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index owner finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index telephoneNumber, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index telephoneNumber finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index uniquemember, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index uniquemember finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index mail, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index mail finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index mailAlternateAddress, size 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index mailAlternateAddress finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index objectclass, size 32768 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example', index objectclass finished, size now 16384 bytes. [10/Jul/2007:14:11:46 +0200] - Repacking backend 'example' ended. [10/Jul/2007:14:11:46 +0200] - Repack finished. [10/Jul/2007:14:11:46 +0200] - Waiting for 6 database threads to stop [10/Jul/2007:14:11:47 +0200] - All database threads now stopped ludo:dsee63 > ll instances/ds1/db/example/ total 1000 drwx------ 2 lpoitou icnc 1536 Jul 3 10:45 . drwxr-xr-x 3 lpoitou icnc 512 Jul 10 14:11 .. -rw------- 1 lpoitou icnc 38 Jul 3 10:45 DBVERSION -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_aci.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_ancestorid.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_cn.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_entrydn.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_givenName.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_id2entry.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_mail.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_mailAlternateAddress.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_mailHost.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_member.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_nsCalXItemId.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_nscpEntryDN.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_nsds5ReplConflict.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_nsLIProfileName.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_nsRoleDN.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_nsuniqueid.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_nswcalCALID.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_numsubordinates.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_objectclass.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_owner.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_parentid.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_pipstatus.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_pipuid.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_pwdaccountlockedtime.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_pwdfailuretime.db3 -rw------- 1 lpoitou icnc 16384 Jul 9 16:38 example_pwdgraceusetime.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_seeAlso.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_sn.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_telephoneNumber.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_uid.db3 -rw------- 1 lpoitou icnc 16384 Jul 10 14:11 example_uniquemember.db3 ludo:dsee63 > ds6/bin/dsadm start /local/demo/dsee63/instances/ds1 Server started: pid=15983 ludo:dsee63 >
Technorati Tags: directory-server, java-enterprise-system, ldap
Ludo Sketches 