Dovecot Pro
2.3
main

Appearance

Updated:

Revision: 174c50a

Note: This is pre-release documentation.
Please access https://doc.dovecotpro.com/latest/installation/upgrading/3.0.x-3.1.x.html for documentation on released versions.

Upgrading from Dovecot Pro 3.0.x to Dovecot Pro 3.1.x ​

Dovecot Pro ​

DANGER

Dovecot v3.1.x Palomar GeoDB schema is incompatible with v3.0.x. To upgrade, you must stop the whole Palomar cluster, recreate the GeoDB schema, and then upgrade all of the Dovecot proxies and backends. This will move users into different backends, so it can be an expensive operation, best done at a time when there is less user traffic. Alternatively you can set up a new v3.1.x cluster and slowly move users between the clusters. You should also take a backup of the v3.0.x schema, so you can easily roll back to v3.0.x in case the upgrade fails.

Required Admin Actions ​

Sieve ​

Global Sieve scripts must be recompiled. See sievec.

Configuration Changes ​

WARNING

Dovecot 2.3.x settings will NOT work unless the configuration is changed as described in this section.

Required Settings ​

The first setting in dovecot.conf MUST now be dovecot_config_version. This helps to avoid unexpected configuration changes in the future.

Another new required setting is dovecot_storage_version. This helps to avoid unexpected storage file format incompatibilities.

Note that the configuration syntax has been changed, and your old configuration will not work without changes.

Configuration Redesign ​

See Dovecot Config File Syntax for the new configuration syntax. This is similar to v2.3, but different in some ways. Especially the configuration is no longer hierarchical - all settings are global settings and can be used anywhere (although they might not actually do anything there). Settings can be inside various filters to specify where they are wanted to be used.

To avoid repetition in setting name prefixes, they are automatically attempted to be prefixed to their parent filter names. For example these are equivalent for the passdb_sql_query setting inside the passdb filter:

doveconf
passdb sql {
  passdb_sql_query = SELECT ...
}
passdb sql {
  sql_query = SELECT ...
}
passdb sql {
  query = SELECT ...
}
passdb sql2 {
  # This will NOT work, as it expands to nonexistent passdb_sql2_query:
  #query = SELECT ...
}

Plugin settings ​

The plugin { ... } section no longer exists. Plugin settings are global the same as all other settings.

Config $variables ​

Using $setting variables in config files requires $SET: prefix now. For example user = $default_internal_user is now user = $SET:default_internal_user. See variable expansion for more details.

Note that earlier it was common to refer to the same setting, e.g.:

doveconf
mail_plugins = $mail_plugins quota

Most of this is now unnecessary, because of the new boollist setting type, which allows you to use:

doveconf
mail_plugins {
  quota = yes
}

Variable expansion ​

A new Settings Variables syntax has been introduced. All old one-letter %variables have been removed, and some old variable aliases have been dropped too.

The new syntax uses | character for "filters" which can be chained. This is why some of the old variables have been replaced by filters. For example the old %d variable is now %{user | domain}, which gets the "user" variable and applies the "domain" filter for it.

List of common short variables and their replacements. ​
Old variableNew syntax
%a%{local_port}
%b%{remote_port}
%c%{secured}
%d%{user | domain}
%h%{home}
%l%{local_ip}
%m%{mechanism}
%n%{user | username}
%p%{client_pid}
%r%{remote_ip}
%s%{protocol} for login & auth settings, %{service} for storage settings
%u%{user}
%w%{password}
Common modifier syntaxes and their replacements. ​
Old syntaxNew syntax
%1Mu%{user | md5 | hexlify(1)}
%2.1Mu%{user | md5 | hexlify | substr(2,1)}
%2.256N%{user | md5 | substr(0, 8) % 256 | hex(2)}
%256Nu%{user | md5 | substr(0, 8) % 256 | hex}
%HuNo conversion available, use one of the syntaxes.
%Ln%{user | username | lower}
%Lu%{user | lower}
%Mu%{user | md5}
Common long variables and their replacements. ​
Old variableNew syntax
%{auth_domain}%{auth_user | domain}
%{auth_username}%{auth_user | username}
%{lip}%{local_ip}
%{lport}%{local_port}
%{mech}%{mechanism}
%{orig_domain}%{original_user | domain}
%{orig_username}%{original_user | username}
%{orig_user}%{original_user}
%{real_lip}%{real_local_ip}
%{real_lport}%{real_local_port}
%{real_rip}%{real_remote_ip}
%{real_rport}%{real_remote_port}
%{rip}%{remote_ip}
%{rport}%{remote_port}
%{service} for login & auth settings (but not storage settings)%{protocol}
Shared namespace variables ​

Previously %%u, %%n and %%d could be used in shared namespace configuration.

These were changed for the namespace_prefix:

Old variableNew syntax
%%u$user
%%n$username
%%d$domain

For other settings, such as mail_path:

Old variableNew syntax
%%u%{owner_user}
%%n%{owner_user | username}
%%d%{owner_user | domain}
%%h%{owner_home}

Directory Hashing ​

If you have been using /home/%2.256N/%u or similar constructs:

  • How to replace %N in new format:

    • %2.256Nu becomes %{user | md5 | substr(0, 8) % 256 | hex(2)} to return maximum 256 different hashes in range 00..ff.

    • %256Nu becomes %{user | md5 | substr(0, 8) % 256 | hex} to return maximum 256 different hashes in range 0..ff (without 0-padding in the front).

  • How to replace %M in new format:

    • %1Mu/%2.1Mu/%u becomes %{user | md5 | hexlify(1)}/%{user | md5 | hexlify | substr(2,1)}/%{user} to returns directories from 0/0/user to f/f/user.

  • There is no way to use %H anymore.

Added Features ​

Settings ​

Changed Features ​

Default Settings ​

SettingOld DefaultNew DefaultNotes
mail_prefetch_count for obox010
mail_fsync for oboxoptimizednever
mail_sort_max_read_count for obox0100
mail_cache_fieldsflagsflags hdr.date hdr.subject hdr.from hdr.sender hdr.reply-to hdr.to hdr.cc hdr.bcc hdr.in-reply-to hdr.message-id date.received size.virtual imap.bodystructure mime.parts hdr.references hdr.importance hdr.x-priority hdr.x-open-xchange-share-url pop3.uidl pop3.order
mail_always_cache_fieldsflags hdr.date hdr.subject hdr.from hdr.sender hdr.reply-to hdr.to hdr.cc hdr.bcc hdr.in-reply-to hdr.message-id date.received size.virtual imap.bodystructure mime.parts hdr.references hdr.importance hdr.x-priority hdr.x-open-xchange-share-url pop3.uidl pop3.order
mail_uidvmail
mail_gidvmail
mail_temp_dir/tmp/dev/shm/dovecot
verbose_proctitlenoyes
import_environment { MALLOC_MMAP_THRESHOLD_ }131072
fs_azure_trailing_slash_workaroundyesnoUp to Dovecot Pro 3.0.3 Azure User SAS and Azure Service SAS based storage authentication made an extra / being appended to the object names. Upgrading from versions smaller than 3.0.3 to 3.1.X will make objects in storage being inaccessible for Dovecot Pro if this workaround is not enabled.
mail_log_prefix%s(%u)<%{process:pid}><%{session}>:%{service}(%{user})<%{process:pid}><%{session}>:New variable expansion syntax
mailbox_list_drop_noselectnoyes\NoSelect folders are now dropped by default.
protocolsimap pop3 lmtpNo protocols are enabled by default.
lmtp_user_concurrency_limitunlimited10The previous conflation of 0 and unlimited is removed, 0 is now an illegal value.

Converted Settings ​

SettingNotes
aclReplaced by acl_driver.
acl_anyoneRenamed to imap_acl_allow_anyone.
auth_debugUse log_debug filter instead. Example: log_debug=category=auth.
auth_default_realmReplaced by auth_default_domain.
auth_policy_server_timeout_msecsReplaced by http_client_request_timeout inside auth_policy named filter.
auth_worker_max_countUse service-specific process limit.
default_idle_killRenamed to default_idle_kill_interval.
disable_plaintext_authReplaced by auth_allow_cleartext.
doveadm_http_rawlog_dirReplaced by http_client_rawlog_dir inside doveadm_server named filter.
event_exporter_format_argsReplaced by event_exporter_time_format.
event_exporter_transportRenamed to event_exporter_driver.
event_exporter_transport_argsSplit off to driver-specific settings, see Event Export.
event_exporter_transport_timeoutReplaced by http_client_request_timeout and event_exporter_unix_connect_timeout.
imap_capability = +CAPA1Use imap_capability { CAPA1 = yes } instead.
inet_listener { address }Replaced by inet_listener_listen setting.
push_notification_backendUse push_notification_driver instead.
mail_crypt_require_encrypted_user_keyReplaced by crypt_user_key_require_encrypted, crypt_user_key_password and crypt_user_key_encryption_key.
verbose_sslUse log_debug = category=ssl instead.
mail_attachment_dirRenamed to mail_ext_attachment_path.
mail_attachment_fsUse fs inside mail_ext_attachment named filter.
mail_attachment_hashRenamed to mail_ext_attachment_hash.
mail_attachment_min_sizeRenamed to mail_ext_attachment_min_size.
mail_attribute_dictUse dict inside mail_attribute named filter.
mail_location setting & mail userdb fieldSplit into multiple mail_* settings.
namespace { location } settingUse mail_* settings inside namespace named list filter.
notify_status_dictUse dict inside notify_status named filter.
notify_status_mailboxUse mailbox_notify_status inside mailbox named list filters.
ftsReplaced by fts named filter.
fts_autoindex_excludeReplaced by boolean property fts_autoindex, nested inside mailbox blocks. Note that the values are inverted as compared to those in the old fts_autoindex_exclude.
fts_decoderReplaced by fts_decoder_driver and fts_decoder_script_socket_path.
fts_enforcedReplaced by fts_search_add_missing and fts_search_read_fallback.
fts_index_timeoutRenamed to fts_search_timeout. The default value is set to 30 secs.
fts_solrReplaced by fts_solr_url, fts_solr_batch_size, fts_solr_soft_commit, and http_client_rawlog_dir settings.
fts_tikaReplaced by fts_decoder_driver and fts_decoder_tika_url.
fts_language_configRenamed to textcat_config_path.
fts_languagesConverted into language blocks.
fts_filtersSplit into language_filters, language_filter_normalizer_icu_id, language_filter_stopwords_dir.
fts_tokenizersSplit into language_tokenizers, language_tokenizer_address_token_maxlen, language_tokenizer_generic_algorithm, language_tokenizer_generic_token_maxlen, language_tokenizer_generic_wb5a, language_tokenizer_kuromoji_icu_id, language_tokenizer_kuromoji_split_compounds, language_tokenizer_kuromoji_token_maxlen.
lazy_expungeRenamed to lazy_expunge_mailbox.
service { idle_kill }Renamed to service_idle_kill_interval.
service { service_count }Renamed to service_restart_request_count. The default value is set to unlimited. Value 0 is now a configuration error.
oauth2 username_formatRenamed to oauth2_username_validation_format.
oauth2 pass_attrsReplaced by oauth2_fields.
oauth2 local_validation_key_dictReplaced by dict inside oauth2_local_validation named filter.
oauth2 timeout_msecs, max_idle_time_msecs, max_parallel_connections, max_pipelined_requests, rawlog_dirReplaced by http_client_* settings inside oauth2 named filter.
oauth2 tls_*Replaced by ssl_client_* settings inside oauth2 named filter.
oauth2 debugUse log_debug filter instead. Example: log_debug=category=oauth2.
oauth use_grant_passwordReplaced by passdb oauth2.
passdb/userdb :protectedRenamed to :default.
passdb { default_fields, override_fields }Replaced by passdb_fields.
userdb { default_fields, override_fields }Replaced by userdb_fields.
pluginAll plugin settings are now global settings.
quota, quota_ruleSplit into separate quota settings.
quota_graceRenamed to quota_storage_grace.
quota_over_flagRenamed to quota_over_status_current.
quota_over_flag_lazy_checkRenamed to quota_over_status_lazy_check.
quota_over_flag_valueRenamed to quota_over_status_mask.
quota_over_scriptReplaced by quota_over_status named filter with execute.
quota_max_mail_sizeRenamed to quota_mail_size. The default value is set to unlimited.
sieveReplaced by storage settings inside sieve_script filter.
sieve_afterReplaced by sieve_script_type = after for sieve_script filter.
sieve_defaultReplaced by sieve_script_type = default for sieve_script filter.
sieve_default_nameReplaced by sieve_script_name.
sieve_discardReplaced by sieve_script_type = discard for sieve_script filter.
sieve_globalReplaced by sieve_script_type = global for sieve_script filter.
sieve_quota_max_scriptsRenamed to sieve_quota_script_count.
sieve_quota_max_storageRenamed to sieve_quota_storage_size.
sieve_user_logRenamed to sieve_user_log_path.
sieve_editheader_forbid_addRenamed to sieve_editheader_header_forbid_add.
sieve_editheader_forbid_deleteRenamed to sieve_editheader_header_forbid_delete.
sieve_spamtest_max_headerRenamed to sieve_spamtest_score_max_header.
sieve_spamtest_max_valueRenamed to sieve_spamtest_score_max_value.
sieve_spamtest_text_value<X>Changed sieve_spamtest_text_value into string list.
sieve_vacation_dont_check_recipientreplaced by sieve_vacation_check_recipient.
sieve_variables_max_scope_sizeRenamed to sieve_variables_max_scope_count.
sieve_variables_max_variable_sizeRenamed to sieve_variables_max_value_size.
sieve_virustest_max_headerRenamed to sieve_virustest_score_max_header.
sieve_virustest_max_valueRenamed to sieve_virustest_score_max_value.
sieve_virustest_text_value<X>Changed sieve_virustest_text_value into string list.
imapsieve_mailbox*See imap-sieve plugin.
ssl_alt_certRenamed to ssl_server_alt_cert_file.
ssl_alt_keyRenamed to ssl_server_alt_key_file.
ssl_caSplit to ssl_server_ca_file and ssl_client_ca_file.
ssl_ca_dirRenamed to ssl_client_ca_dir.
ssl_ca_fileRenamed to ssl_client_ca_file.
ssl_certRenamed to ssl_server_cert_file.
ssl_cert_username_fieldRenamed to ssl_server_cert_username_field.
ssl_client_certRenamed to ssl_client_cert_file.
ssl_client_keyRenamed to ssl_client_key_file.
ssl_dhRenamed to ssl_server_dh_file.
ssl_keyRenamed to ssl_server_key_file.
ssl_key_passwordRenamed to ssl_server_key_password.
ssl_prefer_server_ciphersReplaced by ssl_server_prefer_ciphers.
ssl_require_crlRenamed to ssl_server_require_crl.
ssl_verify_client_certRenamed to ssl_server_request_client_cert.
stats_http_rawlog_dirReplaced by http_client_rawlog_dir inside stats_server named filter.

External Configuration Files ​

v2.3 configured sql, ldap and some other features using .conf.ext external configuration files. These have been replaced by regular settings, which can be used in the same place where they used to be referred to before. For example:

passdb {
  driver = mysql
  args = /etc/dovecot/dovecot-sql.conf.ext
}
connect = host=127.0.0.1 user=mysql_user pass=mysql_pass
password_query = SELECT password FROM users WHERE user = '%u'
# Use these mysql settings globally. These could be also inside passdb sql {}
mysql 127.0.0.1 {
  user = mysql_user
  password = mysql_pass
}
sql_driver = mysql

passdb sql {
  query = SELECT password FROM users WHERE user = '%{user}'
}

Service Settings ​

The default service settings have been changed:

Lua HTTP API ​

Some settings were renamed. See lib-lua: HTTP Functions. The debug setting was removed - use log_debug instead to enable debugging.

Lua Authentication ​

Variable expansion changes affect also auth_request fields. For example auth_request.service is now auth_request.protocol.

Shared Mailboxes ​

If you had SQL or Cassandra based acl_sharing_map, the performance wasn't optimal. A new acl_dict_index setting was added to improve this, which is enabled by default, and needs changes. First, add a new index to SQL/CQL:

sql
CREATE INDEX user_shares_from_user
  ON user_shares (from_user); -- because we search for from_user when rebuilding ACLs

And add a new dict_map for the indexed reverse lookups:

doveconf
dict_server {
  dict acl {
    #dict_map shared/shared-boxes/user/$to/$from {
    # ...
    #}

    dict_map shared/shared-user-boxes-rev/$from/$to {
      sql_table = user_shares
      value_field dummy {
      }

      key_field from_user {
       value = $from
      }
      key_field to_user {
        value = $to
      }
    }
  }
}

Use ACL settings instead of Global ACL Directories or Global ACL File ​

With the following starting configuration:

doveconf
namespace inbox {
  prefix = INBOX/
  separator = /
}

plugin {
  acl = vfile:/etc/dovecot/acls/
}
owner lrwstipekxa
anyone lr
user=kim l
user=timo lr
user=kim lrw
user=kim lrw

Or the same ACLs in a global ACL file:

INBOX owner lrwstipekxa
INBOX anyone lr
INBOX user=kim l
INBOX/foo user=timo lr
INBOX/foo user=kim lrw
INBOX/foo/bar user=kim lrw

You have to create the new ACLs to dovecot.conf:

doveconf
namespace inbox {
  # previously from /etc/dovecot/acls/INBOX
  mailbox INBOX {
    acl owner {
      rights = lrwstipekxa
    }
    acl anyone {
      rights = lr
    }
    acl user=kim {
      rights = l
    }
  }
  # previously from /etc/dovecot/acls/foo/.DEFAULT
  mailbox INBOX/foo {
    acl user=timo {
      rights = lr
    }
    acl user=kim {
      rights = lrw
    }
  }
  # previously from /etc/dovecot/acls/foo/bar
  mailbox INBOX/foo/bar {
    acl user=kim {
      rights = lrw
    }
  }

Note that at this point you could simplify specific rules, e.g. use mailbox name wildcards to replace lines for a specific user:

doveconf
mailbox INBOX/* {
  acl user=kim {
    rights = lrw
  }
}

And re-configure the ACL plugin:

doveconf
acl_driver = vfile
acl_globals_only = yes

Afterwards you can remove the old global ACL directory parent:

sh
rm -rf /etc/dovecot/acls/

Last updated:

Copyright © Open-Xchange GmbH