Palomar: Transition from OX Dovecot Pro 2.3.x Architecture

WARNING

This document describes one method, that has successfully been performed in production systems, for transitioning to Palomar from director based clusters.

Another method that may fit your needs is a migration.

For additional guidance about your transition and your specific needs please contact Open-Xchange for consultation.

All configuration and steps below should be thoroughly tested on a staging or development platform before deployment in production.

Upgrade to latest 2.3.x release

All Dovecot infrastructure should be on the latest release of the 2.3.x branch.

Configure backend and proxy servers

See Palomar Configuration.

At this time make sure that the Passdb configuration on proxy servers contains the host setting. This setting allows the Palomar capable proxy servers to continue to proxy through directors.

Upgrade to 3.0

Dovecot backends and proxies will need to be updated to the 3.0.x version. Make sure that the dovecot-pro-cluster package is installed.

Build director hash to Palomar user mapping

WARNING

Avoid running the migration after backends have recently been added, removed or vhost changed. Also avoid such backend changes after the migration script. Otherwise users' backends may significantly change from the director mappings.

Please also note that director map created in the first step is only valid for a time. That default time is 15 minutes (director_user_expire setting). After this time expired the transition tools fall back to consistent director hashing on their own.

1. Create map of users from director

   director_server:$ doveadm -f json director status > directors.json
   director_server:$ doveadm -f json director map > director_user_map.json

2. Create a list of all users and save it to a file called usrlist.txt (Only needed to process all users at once (4.2a)

3. Gather above lists to a proxy server

4. Populate geodb with users information based on director output and the configured Palomar cluster.

   1. Gather information on the cluster and director setup

   # Decompress required scripts
   proxy_server:$ gunzip /usr/share/doc/dovecot-pro-cluster/scripts/migrate_start.py.gz
   
   # Generate output/configuration file for transition
   proxy_server:$ python3 /usr/share/doc/dovecot-pro-cluster/scripts/migrate_start.py \
     --director-info directors.json \
     --director-map-info director_user_map.json \
     --dns-socket /run/dovecot/dns-client \
     -p -o /tmp/output.json

   2. a) Process all users at once and update information in geodb

   # Decompress required scripts
   proxy_server:$ gunzip /usr/share/doc/dovecot-pro-cluster/scripts/migrate_finish.py.gz
   
   # Populate geodb with user site information (site and backend)
   proxy_server:$ python3 /usr/share/doc/dovecot-pro-cluster/scripts/migrate_finish.py \
     -c /tmp/output.json -f /tmp/extra-users.list \
     --cassandra-nodes <space_separated_list_of_geodb_cassandra servers> \
     --cassandra-keyspace <cassandra_keyspace> \
     --cassandra-batch-size-limit 100

   2. b) Configure cluster_director_transition_config to point to /tmp/output.json for all proxies to set users' target backends. This way geodb will only be updated once users log in. Not for all users provided via /tmp/extra-users.list like in 2a).

   # Place /tmp/output.json on all backends under /etc/dovecot/transition.json
   
   # Configure all proxies to use the transition configuration
   echo "cluster_director_transition_config=/etc/dovecot/transition.json" >> /etc/dovecot/dovecot.conf

5. Check a user to confirm that backend is consistent from director and Palomar. Please note that for Palomar the user need to log in once via Palomar routing. (Alternatively run proxy_server:$ doveadm cluster user access -u <user_id>)

   director_server:$ doveadm director status <user_id>
   proxy_server:$ doveadm cluster user status <user_id>

Enable Palomar

Your Dovecot installation is now ready for Palomar. The next steps will transition your routing from a Director-based architecture to a Proxy-based architecture.

• Remove the host extra field from the proxy Passdb.
• Restart Dovecot.

After some time, the Controller will begin moving users around to balance your cluster. It is also worth mentioning that enabling Dry Run mode for Controller features at this stage can be helpful. This will log the actions the Controller would take without actually performing them.