Dovecot Pro
Search K

Appearance

Dovecot Pro: Getting Started ​

This page is intended as an overview of how to setup and configure a basic, minimal Dovecot Pro Palomar system.

This page is designed to allow system administrators to become familiar with the Palomar architecture and the various Dovecot features, systems, and software packages that are required to accomplish this goal.

IMPORTANT

It is the responsibility of Dovecot Pro administrators to design and then build a configuration for their specific use-case rather than re-using a generic template. This is quick-start "sample configurations" are not provided.

This page should provide all information needed to create this initial configuration to fit your local environment.

Contact Open-Xchange Support if you have questions during your initial configuration process.

Dovecot Pro and Palomar ​

The single most important page to read and understand is the Dovecot Pro Product Description.

Read that page, then read it again.

The product description page describes the various components of Dovecot Pro in general, and the Palomar architecture in more detail. Once you understand all of the concepts on that page, you have the knowledge to determine how to configure your local installation and will understand the particulars presented on the rest of this page.

Palomar Architecture ​

Next, read Palomar Architecture to familiarize yourself with Palomar, the architecture at the core of Dovecot Pro 3.x.

Release Notes ​

Finally, read through the Release Notes for (at least) the latest Dovecot Pro release.

The Release Notes serve as the official source for information on changes and improvements in new versions.

Even if some details seem unfamiliar now, reading these notes now will help you become accustomed to the kind of information provided in future releases, including essential updates that may require your attention.

The Dovecot Team invests a significant amount of time in curating this information to ensure it is valuable to users. The Release Notes are the primary source of knowledge for how the product evolves and grows over time.

Prerequisites ​

External Services ​

You need to setup and populate your user database for authentication purposes.

For mail delivery, you need to have both a SMTP service for inbound mail delivery and a SMTP service for outbound service. Spam and antivirus (AV/AS) filtering are handled in these systems.

If you are running virtual machines, you need to have sufficient resource availability.

Hardware Requirements ​

WARNING

These are not sizing guidelines, but minimum required resources.

These component sizing recommendations reflect what customers have historically used to size their systems. Primarily, this sizing has been driven by available VM sizes available in local environments. It is possible to size components larger and smaller based on the deployment resources - there is no required sizing for any Dovecot component.

For sizing personalized to your local environment, OX Professional Services are available to assist.

A minimum Palomar installation requires the following Dovecot Pro provided components:

  • one Dovecot Pro Proxy
  • two Dovecot Pro Backends
  • one Dovecot Pro Cluster Controller container

These hosts can be virtual machines.

You will require at least three hosts for a Cassandra cluster.

For storage hardware requirements, consult your storage provider's documentation or support.

Proxy ​

Each Proxy should have at least 4 CPU cores and 4 GiB of memory.

Backend ​

Each Backend should have at least 4 CPU cores and 32 GiB of memory.

For metacache you need at least 100 GiB of disk space, 10 GiB of disk space for fs-cache, and 10 GiB for Dovecot Pro Full Text Search (FTS) fs-cache.

Supported Operating Systems ​

See supported operating systems for the rules that control when and how Dovecot Pro releases add and remove support for specific Operating System versions.

Supported Operating Systems for a given Dovecot Pro release are identified in the Release Notes.

Required System Users ​

Dovecot packages automatically set up required user accounts dovenull and dovecot.

In addition, an account for temporary user data is also required. This is typically vmail, which is used in this documentation.

The dovenull account is used for login processes, dovecot for Dovecot internal processes, and vmail for user mail access.

Distinct users, with distinct system privileges, are required for security design reasons.

See system users configuration.

Network Connectivity and Firewall Rules ​

For proper operations, the Proxies need to be directly accessible to end users. The Proxies are located behind transparent load balancers as described in the Product Definition.

The Proxies MUST be able to connect to all Backends available in the local site, and the Cassandra database used for Palomar GeoDB. If running multiple sites, the Proxies MUST also be able to connect to the Proxy pools running on all other sites.

Backends MUST be able to access your storage provider and Cassandra database for BOTH Palomar GeoDB and dictmap.

Proxies and Backends MUST be able to access your userdb and Passdb for user authentication and configuration information.

Cluster Controller MUST be able to access Cassandra database for Palomar GeoDB.

Firewall Rules ​

INFO

This table lists the default ports for ALL services provided by Dovecot Pro. Your system may differ based on your local configuration.

ApplicationDirectionPortExplanation
ProxyInbound110POP3 access, STARTTLS
ProxyInbound143IMAP access, STARTTLS
ProxyInbound465Submission service, SSL
ProxyInbound587Submission service, STARTTLS
ProxyInbound993POP3 access, SSL
ProxyInbound995IMAP access, SSL
ProxyInbound4190ManageSieve, STARTTLS
ProxyOutbound110POP3 to backend
ProxyOutbound143IMAP to backend
ProxyOutbound587Submission service to backend
ProxyOutbound4190ManageSieve to backend
ProxyOutbound9042Cassandra access
ProxyOutbound9142Cassandra access, SSL
BackendInbound110POP3 access
BackendInbound143IMAP access
ProxyInbound465Submission service, SSL
BackendInbound587Submission service
ProxyInbound993POP3 access, SSL
ProxyInbound995IMAP access, SSL
BackendInbound4190ManageSieve access
BackendOutbound9042Cassandra access
BackendOutbound9142Cassandra access, SSL
Cluster ControllerInbound8443Management API
Cluster ControllerOutbound9042Cassandra access
Cluster ControllerOutbound9142Cassandra access, SSL

Storage

Consult your storage vendor for the required ports for storage access.

Time Synchronization ​

Dovecot REQUIRES all servers to have synchronized time.

Ensure the system clock remains accurate by using a time synchronization service such as systemd-timesyncd or ntpd. This prevents issues like Time Moved Backwards Error.

See Time Synchronization.

Dovecot Pro Components ​

Now that you have read and understood the Dovecot Pro software, and have setup the prerequisites, you can begin to install and configure the individual Dovecot Pro components.

This guide suggests one path to accomplish this goal by individually installing and configuring components one at a time. This is a suggestion and not a requirement.

Broader Dovecot concepts (e.g. configuration syntax, repository setup) will be introduced once and then assumed for subsequent sections.

This guide assumes that Dovecot Pro will handle and route IMAP and LMTP. Support for additional protocols is out of scope for this guide.

NOTE

This guide describes configuration of a single Palomar site. Multi-site setups are beyond the scope of this guide.

Proxy and Backend Shared Configuration ​

Dovecot is configured in /etc/dovecot/dovecot.conf. It's possible to place all the configuration into this one file, or if preferred, split it into multiple included configuration files.

See Dovecot Config File Syntax for configuration syntax details.

TIP

You can run doveconf(1) to see how Dovecot has actually parsed the configuration, in case it seems like some change isn't working.

All Dovecot configurations MUST have the following:

See All Dovecot Settings.

Logging ​

See logging for how to configure logging.

Service Configuration ​

Dovecot process limits and session reuse are configured in service setting.

Dovecot Pro contains the recommended configuration for running all services in "high performance" mode by default. It should not be necessary to change the service configuration.

See services configuration for more details.

Statistics and Metrics ​

Dovecot can export internal events. These exposed events are called "metrics". Which metrics are exposed can be configured using Event Filtering.

To enable default metrics:

@metric_defaults = proxy    # For Proxy
@metric_defaults = backend  # For Backend

For more details on metrics support, see Statistics.

OpenMetrics ​

For OpenMetrics support, enable an inet_listener for the stats service:

service stats {
  inet_listener http {
    port = 9900
  }
}
Events Export ​

It is possible to export events using event_exporter. See Event Export for more details.

# Create a JSON log exporter
event_exporter log-export {
  driver = log
  format = json
  time_format = rfc3339
}

# Export the metacache_user_clean_finished metric which is part of
# @metric_defaults=backend using the log-export configured above.
metric metacache_user_clean_finished {
  exporter = log-export
}

Security Best Practices ​

WARNING

This is a non-exhaustive list of security-related topics to review.

Networking / Listening Ports ​
  • listen configures which IP addresses to listen to.
  • login_trusted_networks configures which IP addresses are trusted. Backends must trust local site Proxies' IP addresses. In a multi-site setup, Proxies need to trust connections from remote site Proxies.
Authentication ​

When authenticating between different Dovecot components, it is not a good idea to use auth_allow_cleartext as this will allow anyone from anywhere to authenticate without transport security.

It is better to use login_trusted_networks to limit the locations where plaintext authentication is allowed.

It is also a good idea to consider TLS secured connections between all nodes.

Storing Secrets ​

Passwords and other secrets should be stored in files that only root can read. Typically dovecot.conf is world-readable. Although it could be set readable only by root, this might cause issues in some situations, such as trying to execute dovecot-lda(1) or doveadm(1) as non-root.

One easy way to make secrets safe is to access them via environment variables. For example to store the authentication master password for a Backend:

passdb static {
  fields {
     password = $ENV:MASTER_PASSWORD
  }
}

To securely set the environment variables, use the Environment setting in a systemd drop-in file, which is readable only by root. See the systemd documentation for your OS distribution for further information, or https://www.freedesktop.org/software/systemd/man/latest/systemd.unit.html

Another possibility could be to add secrets to a separate file readable only by root, and:

!include_try dovecot-secrets.conf

With this setting, the include is silently skipped if the file can't be read.

Proxies ​

The Proxy's main function is initial authentication/authorization and user identity normalization. Dovecot supports proxying IMAP, POP3, Submission, LMTP, ManageSieve, and doveadm connections to other hosts.

Proxies directly expose publicly available services and handle initial client connections. They make a user database lookup to the customer's identity management or authentication system (e.g., LDAP) to authenticate and authorize the user, and to lookup user-specific routing parameters.

Proxies are usually configured to handle SSL/TLS encryption, including the SSL certificate management. This may also be done by the external load balancer in front of the Proxies, but STARTTLS commands cannot be used in that case.

Authentication and user identity normalization MUST be done at this layer. Palomar assumes the user has been authorized in all layers below the Proxy.

After authentication, sessions are routed to the site where the user is currently assigned.

The cluster service, which runs on the Proxies, makes sure that a user’s data is not concurrently accessed by multiple Backends. This is required to optimize performance and to avoid seeing stale user data.

Proxies are stateless allowing any of them to be removed or become unavailable without end-user impact. A user's session may be terminated on the client, but this is an expected event in mail access protocols and transparent client reconnection will re-enable the session.

Proxies are connected to:

  • the customer's authentication systems (Passdb/userdb) to authenticate the platform and (optionally) enforce routing decisions.
  • (optionally) OX Abuse Shield to apply authentication policy.
  • Palomar GeoDB to track user routing inside the Palomar platform.
  • Backends in order to route authenticated users to the system that will handle the mail session.
  • external load-balancer layers in foreign sites to route connecting users that are not being serviced by the local site.

The Proxies do NOT need to connect to mail storage, backends in foreign sites, or any other Proxies directly.

Installation ​

Proxies are configured as part of the base Dovecot software.

Required Dovecot Packages ​

INFO

Proxies require knowledge of various Dovecot features for proxying purposes, specifically for doveadm commands.

Thus, packages like FTS are needed even through the proxy itself is not processing any data.

Install Dovecot Pro from the repositories you can find from:

Minimum required package list:

dovecot-pro-core
dovecot-pro-sqlite
dovecot-pro-pigeonhole
dovecot-pro-cassandra-plugin
dovecot-pro-obox
dovecot-pro-cluster
dovecot-pro-fts
dovecot-pro-ldap
dovecot-pro-imapd
dovecot-pro-lmtpd
dovecot-pro-ldap

Configuration ​

INFO

See common configuration section above for information on general Dovecot configuration requirements.

Proxy Authentication ​

Authentication (along with storage) tends to be the most complicated part of a Dovecot configuration, as it directly ties in to a site's existing IDM and user databases.

NOTE

Dovecot does not contain an IDM or other databases to store user information. It is the responsibility of a customer to provide these systems. Dovecot is designed to interface with these existing systems.

See:

Dovecot natively supports a variety of authentication databases (e.g., LDAP authentication). If local authentication requires an unsupported database, or requires customization for the local environment, the Lua authentication database driver should be used.

Proxying to Backends ​

The Passdb must be configured to return proxy=yes extra field. Next, the passdb must specify how to authenticate to Backends. There are two ways to do this:

  1. Forward the user-given password (or OAUTH token) to the remote server. This is done by returning pass=%{password} and proxy_mech=%{mechanism} extra fields.

    • This doesn't work if any non-cleartext, non-token-based authentication mechanisms are used, because they prevent such password forwarding by design.
    • proxy_mech is needed only if both OAUTH and cleartext mechanisms are enabled.

  2. Login to the remote server using a master password. This is done by returning pass=master_secret extra field. This allows client to use also non-cleartext authentication.

Cluster ​

See Palomar Proxy Configuration for configuring Palomar for Proxy, including the monitoring users.

Doveadm ​

Doveadm is a useful protocol for executing Dovecot administration commands. Some of these are "mail commands" that work on user data. The cluster service also executes some of these commands internally. If these commands are run on the proxy, they are automatically proxied to the user's correct backend. See Doveadm Configuration how to configure this.

Doveadm commands can also be configured to be available via Doveadm HTTP API.

HAProxy ​

If HAProxy (or compatible) load balancer is used in front of the Proxies, see Palomar Configuration how to configure it to be trusted.

Mail Plugins ​

Some plugins have doveadm mail commands, which can be proxied to Backends. Enable these commands on Proxies by loading the plugins:

mail_plugins = obox quota fts fts_dovecot

Protocols ​

Configure enabled protocols:

protocols = imap pop3 lmtp submission managesieve

LMTP ​

LMTP service in Dovecot Pro listens by default on port 24.

Cassandra & GeoDB ​

See Palomar GeoDB Configuration.

SSL/TLS ​

See SSL how to configure SSL/TLS for Proxies.

Backends ​

The Backend does the primary work of reading and writing mails to storage and handling the bulk of the mail protocol interaction with the client.

The Backends are organized as a pool of independent nodes. A user is not permanently assigned to a specific Backend. However, due to the performance and load reasons, the platform is designed to allow users to move between Backends.

Backends are connected to:

The Backends do NOT need to connect to Backends on foreign sites, or any Proxies.

Installation ​

Backends are configured as part of the base Dovecot Pro software.

Required Dovecot Packages ​

Install Dovecot Pro from the repositories you can find from:

Minimum required package list:

dovecot-pro-core
dovecot-pro-sqlite
dovecot-pro-lua
dovecot-pro-pigeonhole
dovecot-pro-cassandra-plugin
dovecot-pro-obox
dovecot-pro-cluster
dovecot-pro-ldap
dovecot-pro-lmtpd

Configuration ​

INFO

See common configuration section above for information on general Dovecot configuration requirements.

Authentication ​

The Backend authentication depends on how Proxy has been configured to authenticate against the Backend. See Proxy Authentication above.

See also:

Palomar (Cluster) ​

See Palomar Backend Configuration for configuring Palomar for Backend.

TIP

Remember to configure the monitoring users and userdb extra fields.

Doveadm ​

See Doveadm Configuration how to configure doveadm for Backends.

FTS Settings ​

See Dovecot Pro FTS and how to configure fts-dovecot.

Mail Settings ​

For more details please refer to Obox Configuration.

Mail Plugins ​

Dovecot Pro relies heavily on its plugin system. Load at least these plugins:

mail_plugins = quota obox fts fts_dovecot

For list of all the available plugins, see the "Dovecot Core -> Plugins" and "Pro Plugins" lists on the documentation sidebar.

Quota ​

To stop users from using an excessive amount of storage it is recommended to configure a quota. See quota plugin for more details.

Protocols ​

Configure enabled protocols:

protocols = imap pop3 lmtp submission managesieve

SSL ​

To enable SSL Encryption configure custom Certificate Authorities and other SSL related settings please refer to SSL.

Storage ​

Dovecot storage configuration involves defining how to manage and access email and index data. As Storage backend it is suggested to use a S3 compatible Storage or AWS S3 see AWS S3 and S3 Compatible Storage for more details on the S3 Storage options.

Additionally Dovecot Pro requires a Cassandra cluster which is used via the dictionary functionality in Dovecot Pro. The distributed and scalable NoSQL database is used to support fast listings of directories which are too expensive or too slow when directly asking the S3 storage for it. The feature of supporting the storage with a dict is called Dictmap in Dovecot Pro. See Cassandra for more details on Cassandra and Dovecot Pro. For more details on product description see Dovecot Pro Product Description.

Make sure that the obox plugin is enabled before attempting to configure storage.

mail_plugins {
  obox = yes
}

For more general information on Storage and Storage support please refer to dictmap.

Dictmap Configuration ​
  1. Configure a Cassandra cluster and create the keyspace mails and the needed tables see Cassandra Keyspace/Tables.
  2. Configure Dovecot Pro with a dict-server to use this Cassandra cluster
  3. Provide AWS-S3 or a S3-compatible storage and configure Dovecot Pro to use it.

Configure Dovecot Pro to use Cassandra

# Load suggested default settings for dictmap and configure your Cassndra hosts
@fs_dictmap_defaults = cassandra
cassandra_hosts = 10.2.3.4 10.3.4.5 10.4.5.6

For more Cassandra configuration options please refer to Cassandra configuration.

S3 Storage Configuration ​

There are two storage driver options for S3 aws-s3 and s3. The aws-s3 driver is supposed to be used with AWS-S3 for other S3 compatible storages it is suggested to use the s3 driver

Configuration Example for the aws-s3 driver using IAM configuration:

# BUCKET_NAME: Storage bucket name to use.
# REGION: AWS region to use for storage.
# S3ACCESS: IAM role to use for storage access.
@fs_defaults = aws-s3
fs_s3_url = https://{{BUCKET_NAME}}.s3.{{REGION}}.amazonaws.com/
fs_s3_region = {{REGION}}
fs_s3_auth_role = {{S3ACCESS}}

For more details about IAM configuration please refer to IAM Authentication

Configuration Example for the s3 driver:

# S3_STORAGE_URL: URL to S3 compatible storage
# BUCKET_NAME: S3 Storage bucket name to use.
# REGION: S3 region to use for storage.
# ACCESSKEY: S3 AccesskeyId to use for storage access.
# SECRET: S3 SecretKey to use for storage access.
@fs_defaults = s3
fs_s3_url = https://{{S3_STORAGE_URL}}/
fs_s3_access_key = {{ACCESSKEY}}
fs_s3_secret = {{SECRET}}
fs_s3_bucket = {{BUCKET_NAME}}
fs_s3_region = {{REGION}}
Compression ​

INFO

Compression is enabled by default when using @fs_defaults = s3|aws-s3 with fs_compress_write_method = zstd

metacache {
  fs compress {
  }
  fs dictmap {
    storage_passthrough_paths = full
  }
  fs s3 {
  }
}

See this documentation for supported algorithms and their settings: fs-compress plugin. For some more information on Compression please refer to Compression

Data Encryption ​

It is recommended to use fs_crypt to encrypt mail data in rest at storage.

INFO

Using fs crypt and compress will leave mails uncompressed and unencrypted to local fs cache. This is a designed trade-off between performance and security.

INFO

Encryption is part of the @fs_defaults = s3|aws-s3 and is enabled by adding crypt_global_private_key.

To use fs crypt, You need to generate a keypair using openssl. See Encryption.

crypt_global_public_key_file = /etc/dovecot/public.pem
crypt_global_private_key main {
  crypt_private_key_file = /etc/dovecot/private.pem
}
FTS Storage configuration ​

When Dovecot Pro FTS (Dovecot FTS Plugin, FTS Settings) is configured and enabled the FTS index files are stored in the same storage as mails or index files. The following example uses the s3 driver globally configured.

Configuration Example for storing fts index files using the s3 driver:

# Enable fts and fts_dovecot plugins
mail_plugins {
  fts = yes
  fts_dovecot = yes
}
# Configure a fts storage using s3:
@fts_fs_defaults =  s3

INFO

This relies on having a fs_s3_url and other needed s3 driver configuration configured globally already

Namespace/Mailbox Configuration ​

Enable default INBOX namespace and English language mailbox names with mailbox_special_use flags added:

@mailbox_defaults = english

To configure a non-default Mailbox namespace please refer to Namespaces.

IMAP ​

See IMAP server.

LMTP ​

LMTP service in Dovecot Pro listens by default on port 24. For more details see LMTP Server.

You may want to adjust postmaster_address.

Sieve ​

To enable sieve please refer to Sieve installation.

Cluster Controller ​

The Cluster Controller is the Palomar component that manages a site's cluster state.

Each site requires running one controller, which handles health checks, load balancing, evacuation automation, and REST API access.

While its failure has no immediate impact, it should be restored as soon as possible.

Deployment Methods ​

See Cluster Controller Installation.

Configuration ​

See Cluster Controller Configuration.

Monitoring ​

See Cluster Controller Monitoring.

Troubleshooting ​

If you encounter issues during installation or setup, check the logs for error messages and ensure all dependencies are running.

Common problems and solutions are covered in Palomar Troubleshooting.

Best Practices ​

For optimal performance and scalability, follow best practices for health checks, user management, and more.

A variety of features and recommendations are available at Palomar Administration.

Troubleshooting and Debugging ​

See Troubleshooting and Debugging.

Support (SLA) ​

See Dovecot Pro SLA.

System Testing ​

Once the basic system is deployed, some simple testing can be done to ensure the system is working. You can refer to testing Dovecot installation for more information on how test Dovecot itself. Make sure to use test accounts, it is also very useful to have test accounts in your production system for debugging purposes.

You should test backends and then proxies with same methodology, to ensure everything works as expected.

Once these tests have been done, you should test moving users with doveadm-cluster-user(1) command and ensure that everything still works.

Administration ​

NOTE

This is not a comprehensive administration guide, just a quick collection of links about basic Dovecot Pro administration concepts.

Dovecot administration can be done using Doveadm utility. It can also be configured to provide JSON API, see Doveadm HTTP API.

For example, if you change dovecot configuration, you can use doveadm reload to apply the new config.

User administration ​

Dovecot has no concept of internal users, so there are no user manipulation commands, yet there are some user related commands.

To test user login, you can use doveadm-auth(1) command, and doveadm-user(1) command to find out about users and their attributes.

You can use doveadm-pw(1) to generate passwords for users.

To kick users from system, use doveadm-kick(1) or doveadm-proxy(1) kick.

For obox mailbox format, there is doveadm-obox(1) command to delete user data from system.

Cluster administration ​

You can use doveadm-cluster(1) commands to manage Palomar cluster.

Mailbox administration ​

To manage mailboxes, you can use doveadm-mailbox(1) commands. These allow full control over user mailboxes.

For example, you can use doveadm mailbox list to list all mailboxes for a user.

Mail administration ​

Doveadm has multiple commands for mail administration.

You can use doveadm-search(1) to find mails in user mailboxes, and doveadm-fetch(1) to retrieve information about individual mail messages.

To delete mail messages, you can use doveadm-expunge(1).

Moving mail around can be done with doveadm-copy(1) and doveadm-move(1) functions.

Events ​

Dovecot supports events which can be used for logging and generating metrics. This can be used for example to log failed authentications instead of using verbose or debug logging.

event_exporter log {
  format = json
  time_format = rfc3339
}

metric auth_failures {
  exporter = log
  filter = event=auth_request_finished AND (NOT success=yes)
}

See Statistics, Event Filtering and Event Export.

See Recommended Metrics on which metrics we recommend.

Optimizations ​

Once your system is deployed and successfully, tested, system optimizations can be considered.

See Dovecot Optimizations for a non-comprehensive collection of optimizations to consider.

Congratulations! ​

If you have reached this point, you should have a basic functioning Palomar deployment.

As noted above, there are many more features of Dovecot Pro available that are not covered in this starting guide, so explore the documentation and/or contact your Account Manager for further product information.

Revision: b32b7be

Last updated:

Copyright © Open-Xchange GmbH