Dovecot Pro
Search K

Appearance

Palomar: Administration ​

REST API Reference ​

Site Operations ​

Adding a New Site ​

doveadm cluster site add adds a new site.

This will add a new site, in online status. You can specify site load balancer hostname, which is used when connections are routed from site to site. Tag is optional and will be used to match users and backends to a site shard. UUID should only be used when re-adding a site.

⚠️ TODO

Site name is required. Only provide Site ID in special cases, it is autogenerated by default. Site load balancer is needed if the site has an external load balancer.

Modifying a Site ​

⚠️ TODO

You can edit site name, site load balancer and site tag.

DANGER

The site name must match Dovecot configuration for the respective site, see cluster_local_site. Changing the site name to non-matching value will cause your cluster site to stop operating.

Site Rebalance ​

⚠️ TODO

Cluster controller performs load balancing automatically. If necessary, you can also trigger extra load balancing.

Setting site features ​

⚠️ TODO

Cluster controller allows configuring site features, these are disabled by default.

Backend Health Auto Handling ​

⚠️ TODO

Controls error mitigation features of controller. If disabled, cluster controller will not offline broken hosts, or try to move users from degraded hosts.

Backend Load Balancing ​

⚠️ TODO

Controls automatic load balancing. If disabled, cluster controller will not attempt to move users between backends to balance load.

Metrics Export ​

⚠️ TODO

Controls providing Prometheus metrics. If disabled, the metrics endpoint in cluster controller will not work.

Proxy Operations ​

Adding a New Proxy ​

Proxies within the same site have identical configuration. Adding a new proxy can be done by just launching a new proxy server and adding it into the load balancer's proxy pool.

Dovecot doesn't explicitly keep track of the proxies. However, they usually show up in the proxy_dest_stats table in GeoDB. It may be useful to manually remove those rows eventually.

Removing a Proxy ​

Remove the proxy server from the load balancer's proxy pool. Wait for a while so at least some of the connections finish on their own. Shutdown the server to forcibly disconnect the rest of the connections.

Backend Operations ​

Adding a New Backend ​

doveadm cluster backend add adds a new backend.

You can usually use just --self when adding the host you are on. Site (cluster_local_site) and hostname (cluster_backend_name) are inferred from config. The host will be added in standby mode. UUID should only be used when re-adding a backend.

⚠️ TODO

Host name is required, and must be resolvable by the controller. You cannot add a host by IP address, it must have valid host name. Only provide Backend ID in special cases, it is autogenerated by default.

Changing Backend Status ​

doveadm cluster backend update --status changes a backend status between offline/standby/online.

  • After controller determines that an offline backend is online again, it changes the status automatically to online.
  • standby backends are not automatically touched.
  • Neither offline nor standby backends receive any new connections. Existing connections are not kicked/moved, so normally backends shouldn't be set standby before the users are moved out.

Changing Load-Factor for Backend ​

doveadm cluster backend update --load-factor changes load-factor for a backend.

This can be used to (temporarily) reduce load from backend. Normally, load should always be set to 100. This is primarily used by the controller.

This setting is not accurate, so setting it to 99 or 78 will not have the expected effect.

Recommended values are 100, 50, and 0 for draining the host.

Evacuating Backend ​

You can evacuate a backend by setting its load-factor to 0 with doveadm cluster backend update --load-factor 0. Controller will notice this and start moving users to other backends. There is also the evacuate button to trigger immediate move of all users to other backends.

If you need a backend evacuation without controller, you can use doveadm cluster user batch move backend --count 1000000 to immediately move all users to other backend(s).

WARNING

This command exits without waiting for the moves to complete.

WARNING

Immediate evacuation methods can potentially increase the load on the source backend significantly if many backends are pulling metacache from it at the same time.

After evacuation is complete, shutdown Dovecot to make sure all sessions are gone.

Finally, set the status of the backend to standby to signify that the backend is not usable for connections.

Removing a Backend ​

Backend Overloaded ​

Controller attempts to prevent backend from becoming overloaded by automatically moving users out, but this may not always happen fast enough. To make it happen faster, you can change backend load-factor to a smaller number or even to 0 to remove all users from the backend.

Disabling Automation ​

Disabling Controller Automation ​

⚠️ TODO

Provide non-k8s information here

User Operations ​

Checking User Status ​

doveadm cluster user status shows the current status for a user.

Accessing User ​

doveadm cluster user access accesses a backend as a given user and prints which backend it was.

This command can also cause a connection to the other site. This command is mainly used internally, but may be helpful also for debugging.

Moving Users to Another Backends ​

doveadm cluster user move backend moves the given user to the given backend.

doveadm cluster user batch move backend moves a number of users to the given backends.

WARNING

This command exits without waiting for the move to complete.

Moving User to Another Site ​

doveadm cluster user move site moves the given user to the given site.

WARNING

This command exits without waiting for the move to complete.

Autoscaler ​

Controller supports autoscaling. There are two autoscalers to choose from:

StaticAutoScaler ​

This is the default scaler, and returns number of backends in site.

LuaAutoScaler ​

This scaler calls method get_desired_number_of_backends(site) and expects it to return number of desired backends.

Managing Controller Features ​

Controller offers feature control via the admin UI or HTTP API. The features that can be switched 'on', 'dryrun' or 'off' are:

  • "Backend Health Auto Handling": Automatically take backends down or bring them back up based on backend Z-score explained in Health Check
  • "Backend Load Balancing": Automatically move users between backends to distribute load as explained in Load Balancing
  • "Metrics Export": Collect dovecot statistics and export them to Prometheus. If this feature is disabled, not metric is exported in response to Prometheus scraping of /metrics HTTP endpoint.

Feature settings are recorded in a table (cluster_settings) in GeoDB and are disabled by default. If the table is not present, all features are enabled for backward-compatibility.

To manage feature settings via the admin page, navigate to Site features page from the left pane. Note that you can only change feature settings of the local site via controller admin UI. To manage the features via the controller API, see OpenAPI documentation.

Monitoring ​

⚠️ TODO

Logging ​

⚠️ TODO

Doveadm Commands ​

cluster user batch move status

Man Pagedoveadm-cluster-user(1)

Output the status of the currently running user batch move.

CLI

Revision: 2f44a0c

Last updated:

Copyright © Open-Xchange GmbH