OpenDJ: Enabling the External Change Log on a single server

Yesterday, I described how easy it is to enable Multi-Master Replication between 2 instances of OpenDJ. One of the nice thing with OpenDJ replication, is that all changes are also publicly available (subject to access control) through LDAP under the cn=changelog suffix, also nick-named the External Change Log.

But the command to enable replication and thus the External Change Log requires 2 servers. So what if you want to enable the External Change Log on a single server ?

Note that this is not a fully supported procedure, but is handy for unit testing against the External Change Log. In production environment, you will have multiple servers for high availability and thus the External Change Log will be automatically configured.

Well you cannot use the dsreplication command, but you can configure the OpenDJ instance with the dsconfig utility.

$ bin/dsconfig create-replication-server -h ldap1.example.com -p 4444 \
  -D "cn=directory manager" -w secret12 -X -n \
  --provider-name "Multimaster Synchronization" --set replication-port:8989 \
  --set replication-server-id:2 --type generic
  
$ bin/dsconfig create-replication-domain -h ldap1.example.com -p 4444 \
  -D "cn=directory manager" -w secret12 -X -n \
  --provider-name "Multimaster Synchronization" --set base-dn:dc=example,dc=com \
  --set replication-server:ldap1.example.com:8989 --set server-id:3 \
  --type generic --domain-name example_com

If you want to be able to join this server in a replication topology, you should also create the global administrator’s entry. If you do so, then you will be able to use the dsreplication enable command as illustrated here.

$ bin/dsframework create-admin-user -X -h ldap1.example.com -p 4444 \
  -D "cn=Directory Manager" -w secret12 --userID admin --set password:password

Once enabled, you can read or search the changes with ldapsearch or other LDAP clients :

$ bin/ldapsearch -D cn=directory\ manager -w secret12 -h ldap1.example.com -p 1389 \
  -J "1.3.6.1.4.1.26027.1.5.4:false:;" -b "cn=changelog" '(objectclass=*)'
dn: cn=changelog
cn: changelog
objectClass: top
objectClass: container

# Public changelog exchange control(1.3.6.1.4.1.26027.1.5.4): dc=example,dc=com:0000012fd9bdf863000300000001;
dn: replicationCSN=0000012fd9bdf863000300000001,dc=example,dc=com,cn=changelog
targetDN: cn=a,ou=people,dc=example,dc=com
changeNumber: 0
changes:: b2JqZWN0Q2xhc3M6IHBlcnNvbgpvYmplY3RDbGFzczogdG9wCmNuOiBhCnNuOiBhCmVudH
 J5VVVJRDogNWQzMTNlY2UtYjY4Mi00MDFiLTg2NmYtM2NiZWNlMWNjNTJjCmNyZWF0ZVRpbWVzdGFtc
 DogMjAxMTA1MTAxMTQ5NTZaCmNyZWF0b3JzTmFtZTogY249RGlyZWN0b3J5IE1hbmFnZXIsY249Um9v
 dCBETnMsY249Y29uZmlnCg==
changeType: add
changeTime: 20110510114956Z
objectClass: top
objectClass: changeLogEntry

# Public changelog exchange control(1.3.6.1.4.1.26027.1.5.4): dc=example,dc=com:0000012fd9be46af000300000002;
dn: replicationCSN=0000012fd9be46af000300000002,dc=example,dc=com,cn=changelog
targetDN: cn=a,ou=people,dc=example,dc=com
changeNumber: 0
changes:: YWRkOiBkZXNjcmlwdGlvbgpkZXNjcmlwdGlvbjogTmV3IG9uZQotCnJlcGxhY2U6IG1vZG
 lmaWVyc05hbWUKbW9kaWZpZXJzTmFtZTogY249RGlyZWN0b3J5IE1hbmFnZXIsY249Um9vdCBETnMsY
 249Y29uZmlnCi0KcmVwbGFjZTogbW9kaWZ5VGltZXN0YW1wCm1vZGlmeVRpbWVzdGFtcDogMjAxMTA1
 MTAxMTUwMTZaCi0K
changeType: modify
changeTime: 20110510115016Z
objectClass: top
objectClass: changeLogEntry

Note: the search above uses the “Cookie Control” which is the optimized way to search the External Change Log. The value “;” means that the “cookie” is unknown, and therefore the search starts from the first change. If you want to continue from the last change received, provide the string value that is either in the ChangeLogCookie operational attribute (returned if asked for) or the comment before the change itself.

7 thoughts on “OpenDJ: Enabling the External Change Log on a single server

  1. Mark Haine 17 November 2015 / 12:55

    Hi Ludo,

    I am working with a recent nightly build of 3.0.0 and have tried this to allow testing of change notification for an external app. Unfortunately is is not behaving as I hoped as writing changes to the directory is now rejected.


    ADD operation failed
    Result Code: 53 (Unwilling to Perform)
    Additional Information: The Replication is configured for suffix dc=external,dc=example,dc=com but was not able to connect to any Replication Server

    Can you suggest how I can fix this?

    M

    • Ludo 17 November 2015 / 17:55

      Hi Mark,
      I’ve just tried with a very recent build of OpenDJ 3.0.0 (actually, my development environment), and the procedure still works if the Replication Domain is configured properly.

      In my first test, I used lpmac.local:8989 as the replication-server property inside my replication-domain.
      And the server was producing errors like this:

      [17/Nov/2015:17:46:21 +0100] category=org.opends.server.types.HostPort severity=ERROR msgID=org.opends.messages.replication.51 msg=The hostname lpmac.local could not be resolved as an IP address

      Trying to do an Add against the server, was resulting in the same error as you’ve described:
      Processing ADD request for uid=user.999,ou=people,dc=example,dc=com
      ADD operation failed
      Result Code: 53 (Unwilling to Perform)
      Additional Information: The Replication is configured for suffix dc=example,dc=com but was not able to connect to any Replication Server

      So I used dsconfig to modify the replication domain and set the replication-server property to localhost:8989 (which I know my server will resolve into IP 127.0.0.1).

      And then things worked as expected. I could create an entry, and search the changelog:

      $ ldapsearch -D cn=directory\ manager -w secret12 -p 1389 -b cn=changelog '(&)'
      dn: cn=changelog
      objectClass: top
      objectClass: container
      cn: changelog

      dn: changeNumber=1,cn=changelog
      objectClass: top
      objectClass: changeLogEntry
      changeNumber: 1
      changeTime: 20151117164712Z
      changeType: add
      targetDN: uid=user.999,ou=people,dc=example,dc=com
      changes:: b2JqZWN0Q2xhc3M6IG9yZ2FuaXphdGlvbmFsUGVyc29uCm9iamVjdENsYXNzOiB0b3AKb2
      JqZWN0Q2xhc3M6IHBlcnNvbgpvYmplY3RDbGFzczogaW5ldG9yZ3BlcnNvbgp1aWQ6IHVzZXIuOTk5C
      mNuOiA5OTlVc2VyCnNuOiA5OTlVc2VyCmVudHJ5VVVJRDogMGM0OTBhYTktNWNiZi00ZTlmLTk2Mzgt
      ZmIzZjRhMDdiZmM3CmNyZWF0ZVRpbWVzdGFtcDogMjAxNTExMTcxNjQ3MTJaCmNyZWF0b3JzTmFtZTo
      gY249RGlyZWN0b3J5IE1hbmFnZXIsY249Um9vdCBETnMsY249Y29uZmlnCg==

      • nilesh 07 April 2021 / 06:00

        Hi Ludo,
        We are using opendj 3.0 as an embedded directory server. Its mostly a standalone installation with no replication. However we have configured to enable changelog where the replication is set to connect to the same server. Few of our customers face this issue intermittently. As per the configuration the replication server is mentioned as localhost so that it should not have any issue. At sites where the changelog is not in use, we are able to suggest to remove the replication (changelog) setup. However if changelog is needed, in that case this error props up and does not go away even after restarts. I have found that if I change the server ids, then this error goes away but dont know what would be the repercussions. Could you please guide?

        Here is the extract from the config.ldif file.

        dn: cn=localhost,cn=domains,cn=Multimaster Synchronization,cn=Synchronization Providers,cn=config
        objectClass: ds-cfg-replication-domain
        objectClass: top
        ds-cfg-replication-server: localhost:8989
        ds-cfg-base-dn: dc=example,dc=com
        cn: localhost
        ds-cfg-server-id: 3
        createTimestamp: 20170328151501Z
        entryUUID: 45709310-d33a-4f3d-af92-955510783295
        creatorsName: cn=Directory Manager,cn=Root DNs,cn=config
        ds-sync-generation-id: 48

        dn: cn=external changelog,cn=localhost,cn=domains,cn=Multimaster Synchronization,cn=Synchronization Providers,cn=config
        objectClass: ds-cfg-external-changelog-domain
        objectClass: top
        ds-cfg-enabled: true
        cn: external changelog

        dn: cn=replication server,cn=Multimaster Synchronization,cn=Synchronization Providers,cn=config
        objectClass: ds-cfg-replication-server
        objectClass: top
        ds-cfg-replication-port: 8989
        cn: replication server
        ds-cfg-replication-server-id: 2
        ds-cfg-replication-db-directory: changelogDb
        createTimestamp: 20170328151455Z
        creatorsName: cn=Directory Manager,cn=Root DNs,cn=config
        entryUUID: 20bd1ad9-0172-42ca-a130-62abc46265f6

      • Ludo 07 April 2021 / 08:28

        Hi Nilesh,
        It’s been a while since I last started an OpenDJ 3.0 server. That version does indeed expect a different server-id for every part of replication. Changing it should have any serious impact, however if you are changing it too often on the same server, it may increase the size of meta-data kept in the replication services, changelog files and possibly the data itself.
        You might also want to look at the logs/errors file and logs/replication files for a more detailed diagnostic of the first error on startup. If replication cannot start properly, the server will reject any changes done against the data. But there are multiple reasons for replication not to start properly, and you haven’t specified the exact error message you are seeing on startup.
        I hope this helps.

  2. nilesh3010 19 October 2021 / 08:44

    hi Ludo,
    We are using an external change log on opendj on the base dn “dc=system,dc=backend”.
    We have partitions such as ou=system1,dc=system,dc=backend and ou=system2,dc=system,dc=backend.

    Currently with the changelog, we are having an issue where the change numbers for these partitions are following the same sequence. Our front end system, queries 500 changes at one time on one partition. If there are no changes for that partition for the next 500 change numbers, the front end system wont change the query until, it gets changes and now it gets stuck.

    What we would like to do is to configure different change logs for these partitions so that the change numbering sequence is different?

    I went through the documentation for setting up external change log for different partitions, but was not able to find any thing.

    Do you know any config setting to achieve this? Is this possible on opendj 3.0?

    Thanks in advance for your time.

    • Ludo 21 October 2021 / 16:05

      The Changelog (cn=changelog) is a virtual view on the replication change logs. There is no way to have multiple sequences of numbers based on different naming contexts (partitions). This has been the same in OpenDJ, Sun DS as well as back to Netscape DS!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s