Dovecot Pro
Search K

Appearance

Palomar: Configuration ​

This page describes the Dovecot config necessary to implement the Palomar Architecture.

WARNING

If you are upgrading from OX Dovecot Pro 2.3.x, see Transition from OX Dovecot Pro 2.3.x Architecture.

INFO

It is recommended to use @cluster_defaults = proxy and @cluster_defaults = backend. Using these settings will set these settings by default:

Click to show proxy settings
  auth_master_socket_path = cluster-auth-master
  auth_socket_path = cluster-userdb
  lmtp_proxy = yes
  login_socket_path = cluster
  service doveadm {
    unix_listener doveadm-cluster-http {
      mode = 0600
      type = http
      user = $SET:default_internal_user
    }
  }
  service auth {
    unix_listener auth-master {
      user = $SET:default_internal_user
    }
  }
  service cluster {
    unix_listener cluster-auth-master {
      mode = 0600
    }
    unix_listener cluster-userdb {
      mode = 0666
    }
    unix_listener login/cluster {
      mode = 0666
    }
  }
  dict_server {
    dict cluster-geodb {
      driver = sql
      sql_driver = cassandra
      cassandra {
        keyspace = geodb
      }
      # (dict_maps stripped out)
    }
  }
  cluster_geodb {
    dict proxy {
      name = cluster-geodb
      socket_path = dict-async
    }
  }
  cluster_localdb {
    dict proxy {
      name = cluster-localdb
      socket_path = dict-async
    }
  }
  metric lmtp_rcpt_finished_failure {
    filter = event=smtp_server_transaction_rcpt_finished AND category=lmtp AND NOT error="" AND NOT enhanced_code=5.*
    metric_group_by dest_host {
    }
  }
  metric lmtp_rcpt_finished_success {
    filter = event=smtp_server_transaction_rcpt_finished AND category=lmtp AND error=""
    metric_group_by dest_host {
    }
  }
  metric proxy_session_failure {
    filter = event=proxy_session_finished AND error_code=* AND NOT error_code=proxy_dest_auth_temp_failed AND NOT error_code=proxy_dest_redirected
    metric_group_by dest_host {
    }
  }
  metric proxy_session_success {
    filter = event=proxy_session_established
    metric_group_by dest_host {
    }
  }
Click to show backend settings
  auth_master_socket_path = cluster-auth-master
  auth_socket_path = cluster-userdb
  lmtp_proxy = yes
  login_socket_path = cluster
  service doveadm {
    unix_listener doveadm-cluster-http {
      mode = 0600
      type = http
      user = $SET:default_internal_user
    }
  }
  service auth {
    unix_listener auth-master {
      user = $SET:default_internal_user
    }
  }
  service cluster {
    unix_listener cluster-auth-master {
      mode = 0600
    }
    unix_listener cluster-userdb {
      mode = 0666
    }
    unix_listener login/cluster {
      mode = 0666
    }
  }
  dict_server {
    dict cluster-geodb {
      driver = sql
      sql_driver = cassandra
      cassandra {
        keyspace = geodb
      }
      # (dict_maps stripped out)
    }
  }
  cluster_geodb {
    dict proxy {
      name = cluster-geodb
      socket_path = dict-async
    }
  }
  metacache_pull {
    dict proxy {
      name = cluster-geodb
      socket_path = dict-async
    }
  }
  cluster_localdb {
    dict proxy {
      name = users
      socket_path = metacache-users
    }
  }

Load Balancers ​

Palomar requires a layer of load balancers to balance traffic ingress to the Dovecot Proxy.

Requirements

The load balancer MUST either be transparent (keeps the original client IP visible) or it MUST support HAProxy PROXY V2 protocol.

Round-robin distribution of incoming connections to the pool of available site proxies is a common setup. More advanced traffic-management can be done as well, although specific configuration details for the load balancer is site-dependent and is out-of scope for this documentation.

Example ​

load-balancer: HAproxy configuration example
# haproxy.conf
#
# Sample Configuration for a Palomar load balancing setup using software
# (HAproxy) load balancing
#
# Sample assumptions:
#   - Incoming traffic will be balanced to 2 proxies, located at 192.168.1.10
#     and 192.168.1.11

defaults
    mode                    tcp
    log                     global
    option                  dontlognull
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 128000

listen lmtp
   bind 0.0.0.0:24
   mode tcp
   balance roundrobin
   server hac-dc1-proxy2 192.168.1.11:24 check send-proxy-v2
   server hac-dc1-proxy1 192.168.1.10:24 check send-proxy-v2

listen pop3
   bind 0.0.0.0:110
   mode tcp
   balance roundrobin
   server hac-dc1-proxy2 192.168.1.11:110 check send-proxy-v2
   server hac-dc1-proxy1 192.168.1.10:110 check send-proxy-v2

listen imap4
   bind 0.0.0.0:143
   mode tcp
   balance roundrobin
   server hac-dc1-proxy2 192.168.1.11:143 check send-proxy-v2
   server hac-dc1-proxy1 192.168.1.10:143 check send-proxy-v2

listen sieve
   bind 0.0.0.0:4190
   mode tcp
   balance roundrobin
   server hac-dc1-proxy2 192.168.1.11:4190 check send-proxy-v2
   server hac-dc1-proxy1 192.168.1.10:4190 check send-proxy-v2

listen submission
   bind 0.0.0.0:587 accept-proxy
   mode tcp
   balance roundrobin
   server hac-dc1-proxy2 192.168.1.11:587 check send-proxy-v2
   server hac-dc1-proxy1 192.168.1.10:587 check send-proxy-v2

Dovecot Proxy ​

Dovecot Proxies authenticate the user, convert to internal user ID (if needed), and redirect to the Dovecot Backend.

Installation ​

A Dovecot Proxy requires the dovecot-pro-cluster package.

Configuration ​

Documentation on cluster settings can be found below.

# Name of the proxy's local site
cluster_local_site = DC1

# Include all the required settings for running in cluster proxy mode
@cluster_defaults = proxy
@metric_defaults = proxy

GeoDB / LocalDB ​

See Palomar GeoDB.

WARNING

The GeoDB tables MUST be manually created. See Cassandra.

cassandra_hosts = cassandra-host-3 cassandra-host-2 cassandra-host-1
dict_server {
  dict cluster-geodb {
    cassandra_keyspace = cluster
  }
}

HAProxy ​

If HAProxy (or compatible) load balancer is used in front of Proxies:

  • haproxy_trusted_networks needs to trust the HAProxy servers' IP addresses.
  • inet_listener_haproxy = yes needs to be set for all listeners that accept HAProxy connections, possibly in a different port. For example:
    service imap-login {
  inet_listener imap-haproxy {
    haproxy = yes
    port = 1143
  }
}

Trusted Networks ​

Dovecot Proxy IPs/networks that must be trusted (via login_trusted_networks):

  • Dovecot Proxies in other sites. If user connection is going to a wrong site, the proxy will connect to the correct site's proxy (via the load balancer).
  • cluster-backend-test service connects to the local proxy. This is based on the first IP address in service imap-login { inet_listener }, which defaults to the listen. Typically this is 127.0.0.1 or ::1.
login_trusted_networks = 127.0.0.1 ::1 192.168.1.0/24

Doveadm ​

The Doveadm TCP service must be configured for all Proxies and Backends:

  • doveadm_password must be set to the same value in both the Proxy and the Backend
  • doveadm_port in the Proxy must be set to the same port as the where the backend listens
  • doveadm_ssl can be used to enable SSL/TLS for doveadm proxying. In this case also set inet_listener_ssl = yes for the doveadm listener.
  • For multi-site Palomar operation the doveadm listener must be enabled also on the Proxies.

Example:

doveadm_port = 2300
#doveadm_ssl = yes # If enabled, set also ssl=yes in the inet_listener
doveadm_password = # shared password between all Proxies and Backends

service doveadm {
  inet_listener tcp {
    port = 2300 # same as doveadm_port
    #ssl = yes
  }
}

Monitoring Backends/Users ​

Palomar requires "test" accounts to be configured so that backends can be monitored.

The test username is recommended to use %{backend_host}, which expands to the tested backend. This way if there is a problem with the test user, it affects only a single backend.

Both the proxy and the backend need a passdb, which can authenticate the test users. They could be in the same passdb as the real users, or they can have a separate passdb. In the latter case, it should be placed as the first passdb to catch the test users.

cluster_backend_test_username = probe-%{backend_host}
cluster_backend_test_password = # shared password

# Passdb configuration for monitoring users first:
passdb test-accounts {
  args = /etc/dovecot/cluster-test-accounts.passwd
  fields {
    proxy = y
    password = # shared password
  }
  driver = passwd-file
}

# Passdb for the real users:
#passdb ldap {
#  ...
#}

See:

Dovecot Backend ​

Dovecot Backends perform all work related to protocol commands and interfaces with the storage.

Installation ​

A Dovecot Backend requires the dovecot-pro-cluster package.

Configuration ​

Documentation on cluster settings can be found below.

# Name of the backend (same as in geodb)
cluster_backend_name = hac-dc1-be3
# Name of the backend's local site
cluster_local_site = DC1

# Include all the required settings for running in cluster backend mode
@cluster_defaults = backend
@metric_defaults = backend

GeoDB / LocalDB ​

See Palomar GeoDB.

WARNING

The GeoDB tables MUST be manually created. See Cassandra.

cassandra_hosts = cassandra-host-3 cassandra-host-2 cassandra-host-1
dict_server {
  dict cluster-geodb {
    cassandra_keyspace = cluster
  }
}

Doveadm ​

The Doveadm TCP service must be configured for all Proxies and Backends:

  • doveadm_password must be set to the same value in both the Proxy and the Backend
  • doveadm_port in the Proxy must be set to the same port as the where the backend listens
  • doveadm_ssl can be used to enable SSL/TLS for doveadm proxying. In this case also set inet_listener_ssl = yes for the doveadm listener.
  • For multi-site Palomar operation the doveadm listener must be enabled also on the Proxies.

Example:

doveadm_port = 2300
#doveadm_ssl = yes # If enabled, set also ssl=yes in the inet_listener
doveadm_password = # shared password between all Proxies and Backends

service doveadm {
  inet_listener tcp {
    port = 2300 # same as doveadm_port
    #ssl = yes
  }
}

Monitoring Users ​

Each backend must receive test user logins from proxies. The users can be in the same passdb as the real users, or they can have a separate passdb. In the latter case, it should be placed as the first passdb to catch the test users.

# Passdb configuration for monitoring users first:
passdb {
  args = /etc/dovecot/cluster-test-accounts.passwd
  driver = passwd-file
}

# Passdb for the real users:
#passdb ldap {
#  ...
#}

# Backends use this to avoid moving the test user to other backends
cluster_backend_test_username = probe-%{backend_host}

Depending on the configuration, the test users may also need explicit userdb configuration so they can successfully log in.

See:

Cassandra ​

You need to setup a Cassandra namespace for Palomar.

Cassandra Setup
cql
create table if not exists tags (
  id uuid,
  tag text,
  primary key ((id))
);

create table if not exists sites (
  id uuid,
  name text,
  tag uuid,
  load_balancer text,
  status text,
  primary key ((id))
);

create table if not exists site_reachability (
  src_site_id uuid,
  dest_site_id uuid,
  reachable int,
  primary key ((src_site_id), dest_site_id)
);

create table if not exists backends (
  id uuid,
  site_id uuid,
  load_factor int,
  host text,
  status text,
  status_reason text,
  user_count_approx int,
  last_moved_from text,
  last_moved_to text,
  move_id uuid,
  move_counter_cur int,
  move_counter_max int,
  force_move_id uuid,
  force_move_percentage int,
  force_move_dest text,
  primary key ((site_id), id)
);

create table if not exists backend_stats (
  site_id uuid,
  id uuid,
  key text,
  value double,
  type text,
  primary key ((site_id), id, key)
);

create table if not exists proxy_dest_stats (
  proxy_site_id uuid,
  proxy_host text,
  dest_host text,
  key text,
  value double,
  type text,
  primary key ((proxy_site_id), dest_host, proxy_host, key)
);

create table if not exists users (
  username text,
  metacache_last_host text,
  last_update timestamp,
  primary key ((username))
);

create table if not exists user_sites (
  username text,
  site_id uuid,
  backend_id uuid,
  preferred_site int,
  primary key ((username), site_id),
);

create table if not exists cluster_settings (
  section text,
  site_id uuid,
  backend_id uuid,
  setting_name text,
  setting_value text,
  primary key ((section), site_id, backend_id, setting_name),
);

Settings ​

Palomar Core Configuration ​

@cluster_defaults

Default[None]
ValueGroups Includes
Allowed Valuesproxybackend

Group that expands to various settings that are required for cluster to run in a proxy or backend.

cluster_backend_name

Default[None]
Valuestring

Host name of the backend. This must be the same name as used by doveadm cluster-backend commands.

Cluster: Applies only to Backend.

cluster_backend_test_username

Default[None]
Valuestring
See Also

This setting is used for two purposes:

  1. Username used by proxy for logging into backends to see if it's up or down. %{backend_host} variable expands to the hostname of the backend.
  2. Backends skip this username when doing batch user moves.

Cluster: Applies to both Proxy and Backend.

cluster_director_transition_config

Default[None]
ValueFile

Path to configuration file for director to cluster transition.

The configuration file has to be created by the migrate_start.py script, which is to be used as described in Transition from OX Dovecot Pro 2.3.x Architecture.

If this file is set, the cluster will perform an additional lookup for the user's backend in the current site, using information from the transition configuration. This lookup is only performed if the user does not already have a backend assigned.

This way, users are assigned the same backend at login as they had in the director setup they are being transitioned from. For more information please refer to Transition from OX Dovecot Pro 2.3.x Architecture.

cluster_geodb

Default[None]
Valuestring

Configuration for the globally shared GeoDB dict. This typically points to Cassandra.

Cluster: Applies to both Proxy and Backend.

cluster_local_site

Default[None]
Valuestring

Name of the local site. This must be the same name as used by doveadm cluster-site commands.

Cluster: Applies to both Proxy and Backend.

cluster_localdb

Default[None]
Valuestring
Advanced Setting; this should not normally be changed.

Configuration for the backend-specific local database. This is expected to point to metacache-users socket.

Cluster: Applies only to Backend.

cluster_proxy_check_backends

Defaultyes
Valuestring
See Also

If enabled, this proxy runs checks on backends that are offline to see whether they have come back online. This only updates statistics to allow controller to set the backends online. Backends that are online are not checked, except once at startup.

Cluster: Applies only to Proxy.

cluster_user_move_timeout

Default31secs
Valuetime

If user moving hasn't finished by this timeout, just assume it finished and continue to the next user.

Cluster: Applies only to Proxy.

Cluster Controller ​

See Cluster Controller Settings.

Revision: 09949e1

Last updated:

Copyright © Open-Xchange GmbH