Upgrading to OpenDJ 5.2

This chapter covers upgrade from previous versions.

If the OpenDJ directory server version is older than 2.6.0, you must upgrade your deployment to use at least OpenDJ directory server 2.6.0 before following the procedures in this chapter.

With the migration of OpenDJ project code from Subversion to Git, the upgrade code has changed to no longer rely on Subversion revision numbers.

As a result, upgrade from a nightly build is not guaranteed to work. Upgrade from one release to another works fine, as does upgrade from a release to a nightly build.

As a workaround, rather than upgrading from a nightly build, install a new server alongside the existing server and use replication to bring the new server up to date before retiring the older server.

This chapter includes the following procedures and examples:

Before You Upgrade

  1. Prepare to perform the upgrade procedure as the user who owns the OpenDJ server files.

    Make sure you have the credentials to run commands as the user who owns the server.

  2. (Optional) If OpenDJ directory server runs with Java 7, move to a newer version, at least Java 11, before continuing the upgrade process.

    To move to a newer version, edit the default.java-home setting in the opendj/config/java.properties file, and then run the dsjavaproperties command.

  3. Download the latest release from the GitHub site.

  4. (Optional) If you are upgrading OpenDJ directory server on Windows, and OpenDJ is registered as a Windows service, disable OpenDJ as a Windows service before upgrade, as in the following example:

    C:\path\to\opendj\bat> windows-service.bat --disableService

    After upgrade, you can enable OpenDJ as a Windows service again.

  5. Make sure you perform a full backup of your current OpenDJ installation to revert if the upgrade fails.

    Due to changes to the backup archive format, make sure you stop OpenDJ directory server and back up the file system directory where the current OpenDJ directory server is installed rather than creating a backup archive with the backup command.

To Upgrade to OpenDJ 5.2

Before starting this procedure, follow the steps in "Before You Upgrade".

To upgrade OpenDJ directory server installed from native packages (.deb, .rpm), install the newer package with the system package manager (sudo apt-get install ./opendj_5.2.0-1_all.deb or sudo dnf install ./opendj-5.2.0-1.noarch.rpm). The package stops the running server, runs the upgrade tool as the dedicated opendj user, migrates file ownership under /opt/opendj to that user, and restarts the service (systemd, with a SysV init fallback) if it was running before the upgrade. Back up the installation directory first, as described in "Before You Upgrade".

After the upgrade the server runs as the non-root opendj user. An instance that listens on privileged ports such as LDAP 389 or LDAPS 636 keeps working under systemd, where the service is granted CAP_NET_BIND_SERVICE. On non-systemd hosts that use the SysV init script, grant the capability another way (for example authbind or an iptables redirect) or use ports above 1024, as described in the installation chapter.

OpenDJ directory server backend storage options have changed since OpenDJ 2.6. The underlying implementation is based on an extensible architecture, allowing you to choose the backend storage type when you create a persistent backend for directory data.

This procedure applies when you upgrade from OpenDJ 2.6, retaining the same underlying backend storage. The configuration changes from a Local DB backend to a JE Backend, and the upgrade procedure migrates the underlying backend database. There is no need to export data to LDIF when following this procedure.

The following steps describe how to upgrade OpenDJ directory server installed from the cross-platform (.zip) delivery:

  1. Log in as the user who owns the current OpenDJ server.

  2. Stop the current OpenDJ server.

  3. (Optional) If you have not already backed up the current OpenDJ server, make a back up copy of the directory where OpenDJ is installed.

  4. Unpack the new files from the .zip delivery over the current server files.

    When upgrading from a release before 5.1.2, first delete the stale lib/org.openidentityplatform.opendj.opendj-server-legacy.jar file (or clean out the lib/ directory) before unpacking the new .zip.

    That file is a duplicate library that shipped in older distributions and is no longer part of the delivery, so unpacking the new archive over the existing installation neither overwrites nor removes it. Because the server builds its classpath from lib/*, the leftover jar makes the runtime report the old binary version, and the server fails to start after the upgrade with an error such as The OpenDJ binary version '5.1.1…​' does not match the installed version '5.1.2…​'. Please run upgrade before continuing.

  5. Run the upgrade command, described in upgrade(1) in the Reference, to bring OpenDJ configuration and application data up to date with the new binary and script files that you copied over the current server files.

    By default, the upgrade command requests confirmation before making important configuration changes. For some potentially long-duration tasks, such as rebuilding indexes, the default choice is to defer the tasks until after upgrade. Tasks that are not performed during upgrade must generally be performed after upgrade but before you restart the server.

    You can use the --no-prompt option to run the command non-interactively, with the --acceptLicense option to accept the license terms non-interactively.

    When using the --no-prompt option, if the upgrade command cannot complete because it requires confirmation for a potentially very long or critical task, then it exits with an error and a message about how to finish making the changes. You can add the --force option to force a non-interactive upgrade to continue in this case, also performing long running and critical tasks.

  6. Start the upgraded OpenDJ server.

    At this point the upgrade process is complete. See the resulting upgrade.log file for a full list of operations performed.

    When you upgrade to OpenDJ 5.0 from an OpenDJ 3 or earlier, the upgrade procedure leaves the HTTP connection handler disabled. The newer configuration supports inheritance and subsresources, but is not compatible with the previous configuration. You must rewrite your configuration to the version described in "REST to LDAP Configuration" in the Reference, and then reconfigure the server to use the new configuration. For details, see "RESTful Client Access Over HTTP" in the Administration Guide.

  7. (Optional) If you are upgrading OpenDJ directory server on Windows, and you disabled OpenDJ as a Windows service in order to upgrade, enable OpenDJ as a Windows service again as in the following example:

    C:\path\to\opendj\bat> windows-service.bat --enableService
Upgrading to OpenDJ 5.0

The following example upgrades an OpenDJ 2.6.3 directory server, backing up the current server directory in case the upgrade process fails. In this example, the server properties are updated to use Java 11, and the Local DB backend is migrated to a JE backend:

$ cd /path/to/
$ sed -e "s/default.java-home=.*/default.java-home=\/path\/to\/jdk11/" \
 opendj/config/java.properties \
 > opendj/config/java.properties.new ; \
 mv opendj/config/java.properties.new opendj/config/java.properties
$ /path/to/opendj/bin/dsjavaproperties
$ /path/to/opendj/bin/stop-ds --quiet
... msg=The Directory Server is now stopped
$ zip -rq OpenDJ-backup.zip opendj/
$ unzip -o ~/Downloads/opendj-5.2.0.zip
$ /path/to/opendj/upgrade --acceptLicense

>>>> OpenDJ Upgrade Utility

 * OpenDJ will be upgraded from version 2.6.3.12667 to
 5.2.0.build-hash
 * See '/path/to/opendj/upgrade.log' for a detailed log of this operation

>>>> Preparing to upgrade

  OpenDJ 5.2.0 introduced changes to the JE backend configuration and database
  format. The upgrade will update all JE backend configurations, but will only
  migrate JE backend databases which are associated with *enabled* JE
  backends. It is very strongly recommended that any existing data has been
  backed up and that you have read the upgrade documentation before
  proceeding. Do you want to proceed with the upgrade? (yes/no) [no]: yes

  OpenDJ 5.2.0 changed the matching rule implementations. All indexes have to
  be rebuilt. This could take a long time to proceed. Do you want to launch
  this process automatically at the end of the upgrade? (yes/no) [no]: yes

  OpenDJ 5.2.0 improved the replication changelog storage format. As a
  consequence, the old changelog content of the current replication server
  will be erased by the upgrade. The new changelog content will be
  automatically reconstructed from the changelog of other replication servers
  in the topology. After the upgrade, dsreplication reset-change-number can be
  used to reset the changelog change-number of the current replication server
  to match another replication server. Do you want to proceed with the
  upgrade? (yes/no) [no]: yes

  The upgrade is ready to proceed. Do you wish to continue? (yes/no) [yes]:


>>>> Performing upgrade

  Changing matching rule for 'userCertificate' and 'caCertificate' to
  CertificateExactMatch...............................................   100%
  Configuring 'CertificateExactMatch' matching rule...................   100%
  Replacing schema file '03-pwpolicyextension.ldif'...................   100%
  Removing 'dc=replicationchanges' backend............................   100%
  Removing ACI for 'dc=replicationchanges'............................   100%
  Adding default privilege 'changelog-read' to all root DNs...........   100%
  Adding PKCS5S2 password storage scheme configuration................   100%
  Rerunning dsjavaproperties..........................................   100%
  Updating ds-cfg-java-class attribute in File-Based Debug Logger.....   100%
  Deleting ds-cfg-default-debug-level attribute in File-Based Debug
  Logger..............................................................   100%
  Updating ds-cfg-default-severity attribute in File-Based Error
  Logger..............................................................   100%
  Updating ds-cfg-override-severity attribute in Replication Repair
  Logger..............................................................   100%
  Removing config for 'Network Groups'................................   100%
  Removing config for 'Workflows'.....................................   100%
  Removing config for 'Workflow Elements'.............................   100%
  Removing config for 'Network Group Plugin'..........................   100%
  Removing config for 'Extensions'....................................   100%
  Removing config for 'File System Entry Cache'.......................   100%
  Removing config for 'Entry Cache Preload'...........................   100%
  Removing file '/path/to/opendj/bin/dsframework'.....................   100%
  Removing file '/path/to/opendj/bat/dsframework.bat'.................   100%
  Migrating JE backend 'userRoot'.....................................   100%
  Convert local DB backends to JE backends............................   100%
  Convert local DB indexes to backend indexes.........................   100%
  Convert local DB VLV indexes to backend VLV indexes.................   100%
  Removing file '/path/to/opendj/bin/dbtest'..........................   100%
  Removing file '/path/to/opendj/bat/dbtest.bat'......................   100%
  Removing content of changelog in '/path/to/opendj/./changelogDb'
  directory...........................................................   100%
  Enable log file based replication changelog storage.................   100%
  Replacing schema file '02-config.ldif'..............................   100%
  Archiving concatenated schema.......................................   100%

>>>> OpenDJ was successfully upgraded from version 2.6.3.12667 to
5.0.0.build-hash


>>>> Performing post upgrade tasks

...

>>>> Post upgrade tasks complete

 * See '/path/to/opendj/upgrade.log' for a detailed log of this operation

$ /path/to/opendj/bin/start-ds --quiet
$
To Upgrade the Windows MSI Installation

Before starting this procedure, follow the steps in "Before You Upgrade". Installing the newer .msi performs a major upgrade that replaces the installed program files, so make a full file-system backup of the current installation first.

  1. Stop the current OpenDJ server; if it runs as a Windows service, stop the service with net stop "OpenDJ Server" from an elevated prompt and let the command finish. The installer also tries to stop it, but only succeeds when it is itself running elevated: started by double-click, it cannot, and refuses the upgrade with Windows Installer error 1722 naming the CheckServiceStopped action rather than replacing the files under a running server. It refuses in the same way while the service is still starting, and gives a stop that is already under way 90 seconds to complete.

    A server started with start-ds.bat rather than as a service is refused in the same way, by the CheckServerNotRunning action: it holds the same program files, and the installer will not replace them underneath it. Stop it with stop-ds.bat and let the command finish. The check allows 60 seconds for a stop that is already under way, because a stop command returning is not the same as the server having released its files.

  2. Back up the file-system directory where OpenDJ is installed.

  3. Install the newer package (GUI or silent). The installer detects the existing installation — the location recorded in the registry by a previous x64 package, or the default directory of the older 32-bit package (C:\Program Files (x86)\OpenDJ) — and installs into the same directory, so your configured instance data (config, db, logs) is kept and only the program files are replaced. If the older server was installed in a custom directory the installer cannot detect, select that directory in the wizard or pass it explicitly on the command line: rather than installing a fresh server into the default directory while emptying the old one, the installer refuses to continue whenever nothing has recorded where the old server lives and the directory it is about to install into holds no OpenDJ server — which also catches a mistyped directory name. That refusal also covers an old server that really is installed in C:\Program Files\OpenDJ, because the 32-bit packages recorded no location at all — and that one case the wizard cannot resolve: choosing the default directory in the wizard leaves the installer with the same values it would have had if you had chosen nothing, so pass OPENDJ on the command line instead — it can be given with or without /quiet, so a wizard installation takes it just as a silent one does. The installer further refuses to install into a directory other than the one it detected, unless the directory you name holds an OpenDJ server itself (see the note below): it replaces an installation in place and cannot move one, so uninstall the existing server first if you want it somewhere else.

    C:\> msiexec /i opendj-5.2.0.msi /quiet OPENDJ="C:\path\to\opendj"

    Pass OPENDJ explicitly whenever an unrelated OpenDJ directory tree — a Zip installation, a copy, a decommissioned instance — sits in C:\Program Files (x86)\OpenDJ while the server you are upgrading lives somewhere else and was installed by a package that recorded no location (5.1.x and earlier). Detection can only ask whether that directory holds a server, not which server, so left to itself it adopts the stray tree: the upgrade then replaces the files there while removing the installation you meant to upgrade. Naming the directory settles it — the installer takes an explicitly named directory that holds a server as the one to upgrade, and only refuses the name if that directory holds no server, which is what relocating an installation looks like.

    A registered Windows service survives the upgrade untouched: it names the service wrapper inside the installation directory, which the upgrade replaces in place, so the registration — including a dedicated service account, recovery actions or dependencies you configured — keeps working against the refreshed server. No --disableService/--enableService cycle is needed.

  4. Run the upgrade command, described in upgrade(1) in the Reference, to bring the configuration and application data up to date with the new binary and script files:

    C:\path\to\opendj> upgrade.bat --no-prompt --acceptLicense

  5. Start the upgraded OpenDJ server; if it is registered as a Windows service, start the service again:

    C:\> net start "OpenDJ Server"
To Upgrade a Docker Container

A container started from a newer image over a volume that holds an instance of an older version upgrades that instance itself: before it starts the server, it runs upgrade --no-prompt --force on the volume. Upgrading the container therefore comes down to replacing it with one started from the newer image over the same volume.

  1. Make sure the instance of the container is on a volume.

    The image declares no volume of its own: a container started without a volume mounted at /opt/opendj/data keeps its instance in the container itself, and removing the container deletes the instance. The following command lists the volumes of the container; the instance is on a volume if one of them is mounted at /opt/opendj/data:

    $ docker inspect -f '{{range .Mounts}}{{.Name}} {{.Destination}}{{println}}{{end}}' opendj
opendj-data /opt/opendj/data

    The following steps use the volume opendj-data: use the name that the command printed instead. An empty name is a bind mount of a host directory, which the following command shows. In the next step, stop the container with its first command, then back up that directory instead of running the second one, and mount the directory again in step 3:

    $ docker inspect -f '{{range .Mounts}}{{.Source}} {{.Destination}}{{println}}{{end}}' opendj

    If nothing is mounted at /opt/opendj/data, stop the container and copy its instance to a new volume, here opendj-data, which the following steps then use:

    $ docker stop -t 60 opendj
$ docker cp opendj:/opt/opendj/data/. - | \
 docker run --rm -i -v opendj-data:/data alpine tar -x -C /data

  2. Stop the container, and back up its volume.

    As described in "Before You Upgrade", back up the files of the stopped instance rather than creating a backup archive with the backup command. docker stop gives the server 10 seconds to shut down before it kills it, so give a server with much data more time. The following example writes the volume opendj-data to an archive in the current directory. The archive holds the private keys of the server and the password hashes of its users, so make sure that only the administrators of the server can read it:

    $ docker stop -t 60 opendj
$ docker run --rm -v opendj-data:/data -v "$PWD":/backup alpine \
 sh -c 'umask 077 && tar -czf /backup/opendj-data.tar.gz -C /data .'

  3. Remove the container, and start a new one from the newer image over the same volume, with every other option of the old container: its environment variables, and for a replicated container its --hostname and replication variables as well. In the following example, the old container was started as in "To Run OpenDJ in Docker":

    $ docker rm opendj
$ docker run -d --name opendj --init \
  -p 127.0.0.1:1389:1389 -p 127.0.0.1:1636:1636 -p 127.0.0.1:4444:4444 \
  -e ROOT_PASSWORD=password -e ADD_BASE_ENTRY=--addBaseEntry \
  -v opendj-data:/opt/opendj/data \
  openidentityplatform/opendj:5.2.0

    Images before 5.2.0 ran their health check bound as the root user, while the current image searches the root DSE anonymously. An instance that rejects unauthenticated requests (reject-unauthenticated-requests:true) answers that search with an error, and its container never reports healthy although the server serves. For such an instance, also mount a file holding the password of an account the health check may bind as, and set HEALTHCHECK_BIND_DN and HEALTHCHECK_BIND_PASSWORD_FILE on the new container, as described in the Health check section of the README of the image.

  4. Wait until the container reports itself healthy:

    $ timeout 5m bash -c 'until [ "$(docker inspect -f "{{.State.Health.Status}}" opendj)" = healthy ]; do sleep 5; done'

    The upgrade tasks that take long, such as verifying or rebuilding indexes, are performed before the server starts, since nobody is there to run them afterwards. On a large instance they can take longer than the start period of the health check, so the container may report unhealthy until the upgrade is over, and the command above may give up before: docker logs opendj shows which task is running, and the command can be run again. A container whose upgrade fails does not report itself healthy: docker logs opendj shows why, and the upgrade log /opt/opendj/data/logs/upgrade.log has the details. Starting it again does not get past the failure either: the volume carries the file /opt/opendj/data/.upgrade-pending from before the upgrade until the upgrade, the tasks that run after it included, has succeeded. An upgrade that failed before it recorded the new version is run again on the next start. When a task that runs after the upgrade fails, such as an index rebuild, the new version is already recorded, and the upgrade would find nothing left to do: a container started over the volume then starts the server, logs that the upgrade did not complete, and does not report itself healthy. Revert as follows, or rebuild the indexes that upgrade.log names, as described in "Rebuilding Indexes" in the Administration Guide, then remove the file with docker exec opendj rm /opt/opendj/data/.upgrade-pending and restart the container with docker restart opendj.

To revert, remove the new container, replace the volume with one restored from the backup, and start a container from the older image over it, with the options of the old container. Restore into an empty volume: files that the upgrade added would be left behind in a volume that the archive is extracted over.

$ docker stop -t 60 opendj
$ docker rm opendj
$ docker volume rm opendj-data
$ docker run --rm -v opendj-data:/data -v "$PWD":/backup alpine \
 tar -xzf /backup/opendj-data.tar.gz -C /data

To upgrade replicated containers, upgrade one container at a time, and wait until it reports itself healthy before you go on to the next one, as described in "To Upgrade Replicated Servers".

To Check Stored DNs When Upgrading From a Release Before 5.2.0

Before 5.2.0, the server read a DN only up to the first ;, up to the end of a quoted value, or up to a space after a hex string value, and silently dropped the rest. Since 5.2.0, it reads the whole string: ; separates RDNs as , does (RFC 2253), any other content after an RDN is rejected, and an attribute value may be empty, as in dc=,dc=example,dc=com. Data written by clients before the upgrade can be affected as follows:

  1. A value with DN syntax that a client wrote with ;, such as member: cn=a;dc=example,dc=com, was indexed under the key of cn=a. It now reads as cn=a,dc=example,dc=com, so an indexed equality search for that DN misses the entry until the index is rebuilt. The same holds for a value that was stored while its syntax was not enforced, and that only now parses.

    The upgrade command therefore offers to verify the equality indexes of the attributes that hold DNs, such as member, uniqueMember, owner or seeAlso, in every enabled backend at the end of the upgrade, and rebuilds them under each base DN where they do not match the entries. The verification reads every entry of these backends, so on a large backend it takes a while; the indexes are only rebuilt where it finds a missing key. The default answer is yes, so upgrade --no-prompt performs the verification too, as do the Docker image (see "To Upgrade a Docker Container") and the native packages, which run upgrade --no-prompt --force. If you declined it, run verify-index on these indexes with the server stopped, and rebuild them where it reports errors, as described in "Rebuilding Indexes" in the Administration Guide. Do the same for a backend that the upgrade could not read, such as a JDBC or Cassandra backend whose database was unreachable: the upgrade then warns, names the indexes and the base DN, and goes on, leaving these indexes as they were. A rebuild that the upgrade started and that fails, for instance because the temporary directory is full, fails the upgrade, as the index rebuilds of other upgrade tasks do: such indexes may be left untrusted, so that searches cannot use them, until they are rebuilt. The upgrade then neither verifies nor rebuilds the indexes of the base DNs that come after, as the same cause would most likely make their rebuild fail too: it names them, and you verify them and rebuild them where needed once the cause is fixed.

    A stored DN value with other content after an RDN, such as cn="a"x,dc=example,dc=com, no longer parses: replace it with the DN that was meant.

  2. An ACI whose target, userdn or groupdn DN uses ; between RDNs now names the whole DN, not the part before the first ;. An ACI whose DN has other content after an RDN no longer decodes: the server logs a warning naming the ACI when it loads it, and enters lockdown mode. Correct the ACI, then leave lockdown mode as described in "Preventing Access While You Fix Issues" in the Administration Guide. A global ACI (ds-cfg-global-aci in config.ldif) of this form stops the server from starting instead, with an error that names the ds-cfg-global-aci attribute of the access control handler: correct it in config.ldif while the server is stopped.

The JMX names of monitor MBeans whose RDN value holds characters other than letters, digits and spaces change: these characters are now percent-encoded instead of dropped, so that two monitors no longer share a name. For example, cn-LDAP_Connection_Handler_0000_port_1389 becomes cn-LDAP_Connection_Handler_0%2E0%2E0%2E0_port_1389, and the names of the replication monitors, which hold (, ) and :, change too. Update the JMX monitoring rules, such as exporter rules or checks, that match these names after the upgrade. The SNMP connection handler finds the connection handlers under their new names.

To Upgrade Replicated Servers

The OpenDJ directory server upgrade process is designed to support a rolling (sequential) upgrade of replicated servers.

Do not upgrade all replicated servers at once in parallel, as this removes all replication changelog data simultaneously, breaking replication.

When the topology still has servers before 5.2.0, do not add or rename entries whose DN holds an empty attribute value, such as dc=,dc=example,dc=com or cn=x+sn=,dc=example,dc=com, until every server runs 5.2.0 or later. A server before 5.2.0 cannot read the DN of such a change: it closes its replication connection when it receives the change, receives the same change again when it reconnects, and so stops replicating altogether. Such an entry cannot be imported into a server before 5.2.0 either.

For each server in the replication topology, follow these steps:

  1. Direct client application traffic away from the server to upgrade.

  2. Upgrade the server as described above.

  3. Direct client application traffic back to the upgraded server.

To Add a New Replica to an Existing Topology

Newer OpenDJ servers have updates to LDAP schema that enable support for some new features. The newer schemas are not all compatible with older servers.

When adding a new server to a replication topology with older servers and following the instructions in "Enabling Replication" in the Administration Guide, also follow these recommendations:

  1. Enable replication using the dsreplication command delivered with the new server.

  2. Use the --noSchemaReplication or the --useSecondServerAsSchemaSource option to avoid copying the newer schema to the older server.

    It is acceptable to copy the older schema to the newer server, though it prevents use of new features that depend on newer schema.

  3. If some applications depend on Internet-Draft change numbers, see "To Align Draft Change Numbers" in the Administration Guide.

To Upgrade OpenDJ REST to LDAP Gateway

  1. Rewrite your configuration to work with the new formats described in "REST to LDAP Configuration" in the Reference.

  2. Replace the gateway web application with the newer version, as for a fresh installation.

To Upgrade OpenDJ DSML Gateway

  • Replace the gateway web application with the newer version, as for a fresh installation.