Dovecot Pro 3.0.0

Upgrading Information:

Upgrading from OX Dovecot Pro 2.3.x to Dovecot Pro 3.x

Status

First Customer Shipment	2024-09-04
Support End	TBD
Current Support Status

Release Notes

Previous Version

OX Dovecot Pro v2.3.21.1

This is the first release of Dovecot Pro 3.x.

This release introduces the Dovecot Pro Palomar Architecture as the replacement to the OX Dovecot Pro 2.x Director-based architecture.

Additionally, this release introduces the new Storage Support strategy for Dovecot Pro.

Support for Azure Blob and "S3 Compatible" storages have been added.
All storages must use the obox mailbox format.
All object storage must use fs-dictmap for mailbox metadata storage.

Known Issues

Deploying Dovecot Pro Palomar Controller with docker-compose is experimental.

Dovecot

New Features

DOV-5192: obox: Added "doveadm metacache pull" command. This new command allows to copy metacache from remote backend to local backend.
DOV-5348: Added dict-expire service, which can be used to periodically delete expired dict-sql rows. See https://doc.dovecotpro.com/3.0.0/core/config/service.html#dict-expire.
DOV-5503: obox: If metacache_last_host_dict is configured, obox can automatically attempt to pull metacache from another backend. See https://doc.dovecotpro.com/3.0.0/storage/configuration.html#metacache_last_host_dict.
DOV-5652: Added JA3 client fingerprinting support to dovecot. See https://doc.dovecotpro.com/3.0.0/core/config/ssl.html#ja3-identifier.
DOV-5653: Events can be exported to a local file or unix socket. See https://doc.dovecotpro.com/3.0.0/core/config/events/export.html#transports.
DOV-5738: Added imap_id_received event, which lists received IMAP ID parameters prefixed with id_param_. See https://doc.dovecotpro.com/3.0.0/core/summaries/events.html#imap_id_received.
DOV-6202: obox: Added azure fs driver.
DOV-6578: obox: fs-s3 and fs-aws-s3 can no longer be used without fs-dictmap.
DOV-6784: Added required dovecot_config_version setting, which specifies the configuration version. This will allow changing default settings for new installations without affecting existing installations, unless they also update the version number in this setting. Currently, the only supported version is 3.0.0.
DOV-6785: Added required dovecot_storage_version setting, which specifies the oldest Dovecot version that must be able to read files written by this Dovecot instance. Currently, the oldest supported version is 2.3.0.

Changes

DOV-4920: Default for fts_dovecot_mail_flush_interval is now 10 (was: 0).
DOV-4920: Default for fts_dovecot_max_triplets is now 200 (was: 0).
DOV-4943: obox: Changed metacache UNIX socket listener defaults to "service metacache { unix_listener metacache { mode=0660, group=$default_internal_group } }".
DOV-4960: "doveadm who" output format changed:
- It contains also proxied connections.
- The protocol column was replaced by service column.
- Alternative usernames (user_* passdb fields) are included.
DOV-4960: "doveadm proxy list/kick" are now aliases for "doveadm who/kick"
DOV-5000: Set default lmtp proxy destination port to 24.
DOV-5039: fs-crypt now requires encryption keys by default. Use "maybe" parameter to allow reading and writing unencrypted files.
DOV-5042: Weak password schemes are no longer accepted by default. See https://doc.dovecotpro.com/3.0.0/core/summaries/settings.html#auth_allow_weak_schemes to enable weak schemes.
DOV-5044: Replace auth_debug setting with log_debug = category=auth.
DOV-5045: Replace auth_worker_max_count with "service auth-worker { process_limit }".
DOV-5222: ssl=required now requires TLS for connections from login_trusted_networks as well.
DOV-5222: Auth events: transport=trusted was changed to transport=secured.
DOV-5246: The \NoSelect leaf folders are now automatically deleted by default (i.e. the obsolete mail_location = ...:NO-NOSELECT setting is now enabled by default). To revert to the former behaviour, use KEEP-NOSELECT (see https://doc.dovecotpro.com/3.0.0/core/config/mailbox/mail_location.html#keep-noselect).
DOV-5257: Error wording has been changed from plaintext to cleartext in errors related to logging in without TLS/secure connection.
DOV-5269: The auth_default_realm setting has been renamed to auth_default_domain, which better reflects its function.
DOV-5323: disable_plaintext_auth setting has been renamed to auth_allow_cleartext, with the logic reversed between the settings. See https://doc.dovecotpro.com/3.0.0/core/summaries/settings.html#auth_allow_cleartext.
DOV-5366: Attempting to use push_notification_backend now gives a warning about the setting being obsolete. Use push_notification_driver instead.
DOV-5407: When calling doveadm mail commands, one of "-u", "-A", or "-F" must be specified. Environment variable USER is no longer accepted as default.
DOV-5408: Enable support for the IMAP COMPRESS extension by default. Relevant plugin settings have been dropped and the imap-zlib plugin has been removed.
DOV-5413: The zlib-save plugin has been renamed to mail-compress. Likewise the relevant configuration items zlib_save and zlib_save_level have been renamed to mail_compress_save and mail_compress_save_level respectively.
DOV-5415: Unknown/invalid "%{variables}" were ignored in some situations, other than becoming expanded into "{UNSUPPORTED_VARIABLE_name}". Now they're causing proper errors instead, which may cause, e.g., authentication failures if the broken behavior was relied on.
DOV-5514: Enable imapc features "delay-login", "search", "fetch-headers", "fetch-bodystructure", "fetch-size" by default. Enable "acl" and "modseq" by default, if the remote server supports it. Replace the appropriate imapc_features with ones that disable features selectively.
DOV-5539: obox: Changed default for max-parallel-iter to 10. See https://doc.dovecotpro.com/3.0.0/storage/dictmap.html#dictmap-parameters.
DOV-5562: Event filters warn about type mismatches when filtering for numeric values.
DOV-5591: Prevent using fscache with ZFS, since it doesn't work correctly.
DOV-5602: Allow non-standard special-use-labels in "mailbox { special_use }" setting, but log a warning about them at startup.
DOV-5630: Renamed proxy_session_finished event's "idle_secs" field to "idle_usecs" to make all event intervals microsecond-based and allow using interval units properly.
DOV-5642: The doveadm HTTP API now requires valid boolean values. Providing invalid boolean values will result in a 400 response.
DOV-5747: userdb fields can be set to empty value now. Previously they became changed to "yes" value.
DOV-5785: bytes_in and bytes_out fields in events have been renamed to net_in_bytes and net_out_bytes respectively.
DOV-5972: Lua passdb/userdb now passes all args key/values to an initialization function. See https://doc.dovecotpro.com/3.0.0/core/config/auth/databases/lua.html#initialization.
DOV-6100: Sub-metric labels are now limited to 256 bytes in total length.
DOV-6128: All sections must now have a (unique) name in the configuration file. This especially means namespace, passdb and userdb sections. For example "passdb { ... }" must now be "passdb somename { ... }".
DOV-6284: auth service's log line prefixes were changed.
DOV-6369: Dovecot's Lua HTTP client no longer sends "Except: 100-Continue" header unless asked to.
DOV-6503: "Maximum number of connections from user+IP exceeded" failure is now returned with "[LIMIT]" instead of "[UNAVAILABLE]" IMAP response code prefix.
DOV-6520: User kicking is now logged as "User kicked" instead of "Server shutting down". Also, there is no "Killed with signal 15" warning logged.
DOV-6588: IMAP hibernation is now enabled by default.
DOV-6599: "name" response field from IMAP ID command always returns "Dovecot Pro".
DOV-6653: Login log format now includes target host, ip, and port as prefix in login proxying.
DOV-6893: Changed some login/disconnection related log lines to make it easier to understand and search them. For example, "Login:" is now "Logging in:" and pre-login failures have now "Login aborted:" prefix instead of "Disconnect:".
DOV-6902: Dovecot Lua HTTP Client will error out if invalid setting name is used.
DOV-6913: "doveconf" without parameters now outputs the same as with "-n" parameter. Use "-a" to get the old behavior (all settings).

Removed Items

DOV-4678: Memcached dict support was removed
DOV-4960: Removed ipc service. Its main functionality was moved to anvil service.
DOV-4994: Removed global ACL directory support. Use global ACL files instead. See https://doc.dovecotpro.com/3.0.0/installation/upgrading/2.3.x-3.x.html#use-global-acl-files-instead-of-global-acl-directories.
DOV-4998: Legacy ssl-parameters.dat conversion support was removed.
DOV-5007: Dirsize quota driver was removed.
DOV-5011: "dsync" command symlink was removed. Use "doveadm sync" or "doveadm backup" directly instead.
DOV-5013: Option "SSLv3" in ssl_min_protocol was removed.
DOV-5016: login_access_sockets setting was removed, including tcpwrap support.
DOV-5017: old-stats plugin was removed.
DOV-5017: imap-old-stats plugin was removed.
DOV-5023: metacache_disable_merging setting was removed. Use metacache_index_merging=none instead.
DOV-5024: Nonstandard X-CANCEL and X-STATE IMAP commands were removed.
DOV-5028: "v1" option in metacache_index_merging has been removed.
DOV-5030: checkpassword passdb/userdb was removed. Use Lua passdb/userdb instead.
DOV-5032: Quota dict driver was removed.
DOV-5036: obox_allow_inconsistency setting was removed.
DOV-5037: "doveadm batch" command was removed.
DOV-5038: Shadow passdb driver was removed.
DOV-5040: sieve: Removed notify, imapflags, and vnd.dovecot.duplicate extensions.
DOV-5043: Remove deprecated wiki documentation included in distribution packages. Replace references to obsolete wiki with new docs.
DOV-5101: license_checksum setting was removed. A notice about the obsolete setting will be raised if it is found in configuration.
DOV-5254: sieve: Removed sieve_vacation_max_subject_codepoints setting.
DOV-5358: auth_client_cache_flush_started and auth_client_cache_flush_finished events were removed.
DOV-5404: The mailbox-alias plugin was removed.
DOV-5535: Director component has been removed.
DOV-5585: Deprecated sieve_dir, sieve_global_path, and sieve_global_dir settings were removed.
DOV-5585: Drop undocumented handling of draft 'import'/'export' of the sieve include extension, which have been replaced by the ':global' command when finalizing the appropriate RFC (RFC 6609).
DOV-5612: obox: HTTP Storage options that have been deprecated since 2.3 have been removed.
DOV-5738: Support for imap_id_log setting was removed. The IMAP ID parameters can now be exported via the imap_id_received event.
DOV-6699: Dict authentication support has been removed. Use Lua based authentication instead.
DOV-6770: Support for IMAP SETQUOTA command to change quota limits was removed. This removes the quota_set setting.
DOV-6952: cassandra: ssl_verify=cert-dns was removed. This behavior is vulnerable to MITM attacks, so it's not safe to use.

Improvements

DOV-3970: If backend IMAP server returns "NO [REFERRAL imap://url/]", IMAP proxy will automatically reconnect to that server instead (unless it detects a loop).
DOV-3981: LMTP supports returning Dovecot-specific referrals now, which Dovecot LMTP proxy will follow. This happens when passdb returns "host" extra field without "proxy" field.
DOV-4142: obox: fs_object_write_uncertain event is sent when a write HTTP operation is timed out.
DOV-4754: Support Write-Ahead Log (WAL) and read-only databases with SQLite driver.
DOV-4901: Added base64url format to hash modifier in variable expansion. For example: "%{sha1;format=base64url:username}".
DOV-4921: Reduce the number of file accesses by FTS on first fts cache access.
DOV-4964: obox: If obox_fetch_lost_mails_as_empty=yes is used and a mail is failing with "object exists in dict, but not in storage" error, don't assume that cached records are corrupted because they don't match the empty mail content. This may be only a temporary situation.
DOV-4997: fts_autoindex_exclude now honors namespaces. See https://doc.dovecotpro.com/3.0.0/core/plugins/fts.html#fts_autoindex_exclude.
DOV-5020: mail_crypt_require_encrypted_user_key now accepts boolean values.
DOV-5053: "doveadm kick" stops any active indexer-workers for matching users (but doesn't remove them from indexer queue).
DOV-5053: Added "doveadm indexer add/remove/list" commands. The remove command especially is useful to remove the user's indexing requests from the queue.
DOV-5054: "doveadm kick" now supports kicking hibernated imap connections.
DOV-5097: Mail processes stop now more quickly when receiving SIGTERM, for example due to "doveadm kick".
DOV-5098: Folder deletion now expunges mails in batches of 1000 mails. This helps especially with lazy-expunge plugin when expunging a lot of mails (e.g. millions) to make sure that the progress always moves forward even if the process is killed.
DOV-5099: Added submission_add_received_header setting to protect sender identity by suppressing the Received: header.
DOV-5191: Added login_socket_path setting, which makes it easier to override the default "login" socket. The old method of providing this as a parameter to the login executable still works, and can be used to override this setting.
DOV-5254: sieve: The maximum length of the subject header for a notification message is now 256 Unicode code points rather than 256 octets, which allows languages using many multibyte characters (like Japanese) to have long subjects as well.
DOV-5401: "doveadm log errors" now shows microseconds in timestamps.
DOV-5418: The "address" Sieve test now allows evaluating the contents of the X-Original-From header.
DOV-5496: Auth policy reports can now include an attribute describing why authentication failed. This attribute is sent as fail_type in defaults. See https://doc.dovecotpro.com/3.0.0/core/config/auth/policy.html#authentication-policy.
DOV-5562: Allow size units when specifying event filter values, e.g., "100kb" instead of "102400".
DOV-5630: Allow interval units when specifying event filter values, e.g. "1min" instead of "60000000".
DOV-5662: Added support for ARGON2 password scheme.
DOV-5667: Configuration of service listener type can now be performed explicitly by setting the "type" field of the listener. Before, (a suffix of) the name or path of the listener was used to determine the listener type, which was error-prone and cumbersome.
DOV-5693: sieve: Added untagged IMAP response code * OK [INPROGRESS...] ... for client keepalive and update progresses in FILTER command, sent every 10 seconds.
DOV-5719: Added login_aborted event. See https://doc.dovecotpro.com/3.0.0/core/summaries/events.html#login_aborted.
DOV-5720: Support SCRAM-SHA-1 and SCRAM-SHA-256 SASL mechanisms for outgoing connections, such as proxying, imapc, and pop3c.
DOV-5744: Added untagged IMAP response code * OK [INPROGRESS...] ... for client keepalive and update progresses in long operations. Where applicable, it replaces the previous * OK Hang in there.. response and it is sent every 10 seconds. SORT and THREAD commands support this only for the searching part, not the sorting/threading.
DOV-5761: Improve error messages when sqlite fails due to permission issues.
DOV-5772: The doveadm command now processes a "--help" argument to print the relevant list of subcommands or the usage of a specific subcommand depending on placement.
DOV-5785: Added net_in_bytes and net_out_bytes fields to mail_user_session_finished event.
DOV-5785: Added pop3_command_finished event.
DOV-5791: Added optional dict proxy parameters idle_timeout and slow_warn.
DOV-5936: "doveadm save": Added -r received-date parameter.
DOV-5973: auth: Added "%{ldap_multi}" variable to handle multi-valued attributes. See https://doc.dovecotpro.com/3.0.0/core/config/auth/databases/ldap.html#ldap-specific-variables.
DOV-5990: Now imapc supports remote GETMETADATA (except on private attributes for users different from the owner).
DOV-5999: Event filters now support escaping wildcard '*' and '?' characters by prefixing them with '\'.
DOV-6022: obox: If a write to Cassandra fails with uncertainty, Dovecot now attempts to clean up by default. First it tries to delete the uncertainly written dict key. If that succeeded the deletion of the corresponding storage object is also attempted. See https://doc.dovecotpro.com/3.0.0/storage/dictmap.html#uncertain-writes. This behaviour can also be disabled by using "no-cleanup-uncertain" as a dictmap parameter. See https://doc.dovecotpro.com/3.0.0/storage/dictmap.html#dictmap-parameters.
DOV-6063: Added mail_metadata_accessed event.
DOV-6210: "doveadm fs put" can now put metadata.
DOV-6266: Improve Cassandra debug and error messages by adding used consistency. For errors add "N of minimum M responses received".
DOV-6377: mail-crypt and fs-crypt now support X25519 and X448 curves.
DOV-6418: Provide more granular error codes in proxy_session_finished event, allowing differentiating between temporary and permanent authentication failures (e.g. max connections vs. incorrect password). See https://doc.dovecotpro.com/3.0.0/core/summaries/events.html#proxy_session_finished.
DOV-6442: Added metacache_forced_refresh_interval setting (default 8h). If user indexes haven't been refreshed for that long, force the refresh even if metacache_close_delay setting otherwise would prevent the refresh. This makes sure that indexes for very active users are still refreshed once in a while.
DOV-6486: Added support for ChaCha20-Poly1305 encryption algorithm.
DOV-6604: fts_message_max_size setting can be used to set the maximum size of message body to be processed for FTS indexing. The rest of the message is ignored.
DOV-6616: Added log_retries=yes parameter to Cassandra connect string. This enables logging every time the Cassandra driver retries a failed request internally.
DOV-6616: Adds "consistency" and "error_consistency" fields to sql_query_finished event.
DOV-6655: intercept: Support UUID matching between X1 and X3.
DOV-6698: Added "--no-userdb-lookup" parameter to doveadm mail commands.
DOV-6752: Added quota_mailbox_message_count setting to limit maximum number of messages for a folder. The default is unlimited.
DOV-6753: Added quota_mailbox_count setting to limit the maximum number of mailboxes a user can create.
DOV-6919: Added auth_internal_failure_delay setting, defaulting to 2 seconds. If auth lookup fails with internal error, the reply to the client is delayed by this much + random 0..50% This is intended to prevent clients from hammering the server with immediate retries.

Fixed Issues

DOV-4819: obox added 128bit GUIDs to dovecot.index.cache, which is unnecessary since they also exist in dovecot.index.
DOV-4830: Header name lengths or counts weren't limited in dovecot.index.cache, making it possible for users to cause Dovecot to behave inefficiently. Now the header name lengths are truncated to 100 characters and a maximum of 100 different headers can be cached at a time per folder. If header count limit is reached, mail_cache_decision_rejected event is sent.
DOV-4849: Using dsync with "tcp" target didn't locally show all log messages sent by remote.
DOV-4986: "doveadm fetch date.sent" reported a time which is incorrect for the represented timezone.
DOV-4995: Sieve: The "string" test from the "variables" extension uses the "i;octet" comparator by default, which is not standard. It must be "i;ascii-casemap" instead.
DOV-4999: Folder renaming was broken when using mail_location's FULLDIRNAME together with INDEX, INDEXPVT, INDEXCACHE or CONTROL. Renaming a parent folder lost the index/control files for all of its child folders, which with some mailbox formats lost the mails until "doveadm force-resync" was used.
DOV-5118: oauth2: JWT tokens containing decimal parts in NumericDate fields were rejected as invalid.
DOV-5219: Could not use IPv6 link-local addresses in listen setting.
DOV-5222: Submission service didn't write the SSL "S" flag correctly to Received: headers. In proxying setups it indicated only whether the last proxy-backend hop was SSL secured, rather than whether the client-proxy connection was SSL secured.
DOV-5226: submission proxy: IPv6 addresses were missing IPV6: prefix in proxy XCLIENT command, which was incompatible if submission proxy connected directly to Postfix (which isn't recommended).
DOV-5242: Setting login_trusted_networks to an invalid value would crash login processes on client connect.
DOV-5247: Log warning: "Server-side warning: USE <keyspace> with prepared statements ..." polluting the log files. The way cassandra tables are accessed has been modified in order to conform with official cassandra recommendations.
DOV-5293: "doveadm mailbox cache remove" crashes when trying to access nonexistent mailbox.
DOV-5320: auth: Fix handling passdbs with identical driver/args but different mechanisms/username_filter. The second passdb was handled identically to the first one, with the first one's mechanisms/username_filter.
DOV-5332: imap-acl: Using IMAP ACL commands (e.g. MYRIGHTS) for a shared mailbox of a nonexistent user would lead to a crash.
DOV-5333: It was not possible to use virtual mailboxes as imapsieve COPY/APPEND destination mailbox. It is still not possible to use the virtual mailbox names themselves (e.g. "Virtual/All"), but using the physical save destination mailbox name works now also via virtual mailboxes.
- For example if "Virtual/All" is configured to save to INBOX, imapsieve_mailbox_name = INBOX will trigger both when saving to INBOX and when saving to "Virtual/All".
DOV-5347: fs-fts-cache: When expunging created a new FTS triplet with increased "generation number", but failed to delete the old FTS triplet, this lead to broken behavior. It may have also resulted in a crash.
- Fixes: Panic: file fs-fts-cache-sync.c: line 426 (fs_fts_sync_find_last_cached_triplet): assertion failed: (i >= 3)
DOV-5377: mail_cache_fields=body.snippet prevented parallel obox writes (mainly in dsync)
DOV-5401: "doveadm log errors" command didn't show errors logged by the master process.
DOV-5416: Aborting some doveadm commands with Ctrl-C may have caused crashes or other bad behavior.
DOV-5472: Invalid parameters to IMAP APPEND command can crash the imap process.
DOV-5480: Using doveadm "-F" parameter to specify a list of usernames didn't work in a doveadm proxy.
DOV-5481: doveadm with the "-F" argument would always attempt to list all users. Now it only matches usernames exactly.
DOV-5509: obox: Message sizes were unnecessarily written to dovecot.index.cache, even though they were also written to dovecot.index.
DOV-5551: fts-dovecot: Corrupted fts.X and fts.Y files weren't always deleted, resulting in repeating errors logged about them.
DOV-5643: The HTTP server component does not correctly log whether HTTP requests arrived on a secure connection.
DOV-5649: Event filter with "duration" field never matched.
DOV-5649: Event filters could not use 0 with non-equal comparison (e.g. >= 0).
DOV-5649: Event filters comparing number type field to a string value (e.g. "bytes_in=foo") crashed with signal 11.
DOV-5654: IMAP URLAUTH extension's URLFETCH command behavior was not RFC compliant.
DOV-5668: "%{userdb:field}" expansion didn't work for certain settings: sendmail_path, submission_host, rawlog_dir, and submission_relay_rawlog_dir.
DOV-5670: fts indexing assert-crashes when snowball filter causes a word to be dropped, e.g. "H." in French.
- Fixes: Panic: file fts-filter.c: line 137 (fts_filter_filter): assertion failed: ((*token)[0] != '\0')
DOV-5680: Process would crash with DNS caching enabled if lookup failed during background refresh.
DOV-5696: SCRAM-SHA-1 and SCRAM-SHA-256 schemes ignored the -r rounds parameter with "doveadm pw".
DOV-5698: obox: Error message is written wrong when index diff bundle write succeeds, but the old diff bundle deletion fails. It likely logged "Asynchronous operation in progress" rather than the real error.
DOV-5703: Auth process crashes (or other communication errors with it) resulted in a regular "Authentication failed" error rather than "internal error".
DOV-5743: If index bundle list cache wasn't up-to-date, accessing the user/folder may have crashed.
- Fixes: Panic: file metacache-get.c: line 542 (metacache_bundle_unpack_with_bundle): assertion failed: (file->base_file->exists_as_file)
DOV-5751: Using email-address fts tokenizer could result in excessive memory usage with garbage email input. This could cause the indexer-worker processes to fail due to reaching the VSZ memory size limit. This was only partially fixed by DOV-4745 / v2.3.18.
DOV-5788: In some rare cases an HTTP client request can hang indefinitely while sending the request payload.
DOV-5793: Object storage failures can cause FTS to crash during expunging.
- Fixes: Panic: cache missing when deleting failed triplets
DOV-5814: The IMAP URLAUTH (and Submission BURL) capability is broken in several ways and causes panics and errors upon URL resolution.
DOV-5822: IMAP URLAUTH URL creation and retrieval fails when the user name is translated to, e.g., some internal GUID upon login. This translation is invisible to clients, which causes URLAUTH URLs produced by clients to be rejected by the server.
DOV-5823: submission: Signal 11 crash if connection is closed by MTA too early.
DOV-5844: Fix validation of characters in IMAP tags and atoms. The DEL character (0x7F) is now rejected in both tags and atoms, and the close square bracket "]" is now accepted in tags (but not in atoms)
DOV-5848: doveadm crashed when invoking "doveadm fts expand -A".
DOV-5856: IMAP NOTIFY events caused imap_command_finished events to be sent with NOTIFY-CALLBACK command name. These shouldn't have been sent.
DOV-5877: dict-fs could have been endlessly looping when iterating in some situations.
DOV-5954: ManageSieve did not emit mail_user_session_finished event.
DOV-5971: IMAP NOTIFY: If a change happened within 1 second after any IMAP command was finished, the notification wasn't sent until some other change/command occurred.
DOV-6002: auth failed to mark the anonymous user as such.
DOV-6005: submission service could have logged sensitive authentication data with debug logging enabled.
DOV-6007: Applying imapsieve scripts to read-only source mailboxes could crash IMAP session
DOV-6061: The "nodelay" passdb extra field has been broken since v2.2.0.
DOV-6062: sieve-filter: Event recursion could have caused the log message to repeat the log prefix and overwrite parts of the actual message.
DOV-6100: Missing field would stop sub-metric "group_by" from first unknown field.
DOV-6125: Doveadm would crash if ran from a process under master process.
DOV-6170: obox: If obox_fetch_lost_mails_as_empty=yes is used and a objects metadata read is failing with "object exists in dict, but not in storage", Dovecot could have crashed after trying fall back reading vsize from index files while vsize was not in the index.
- Now the object is marked as broken which prevents the reading.
- Fixes: Panic: file obox-mail.c: line 486 (obox_mail_get_virtual_size): assertion failed: (data->virtual_size != (uoff_t)-1)
DOV-6179: Auth service crash can cause login services to crash too on a busy system. This change will cause authentications to fail instead.
DOV-6210: metawrap with compress, crypt, and posix fs drivers would fail.
DOV-6212: MTA policy server would get badly treated with multiple recipients.
DOV-6214: If oauth2 response failed with 200 OK, but the response JSON had error field, dovecot would not process this as error.
DOV-6236: Fixed case where dovecot-sysreport stops silently if gdb is not installed and core flag is used. Now the error is reported properly.
DOV-6239: obox: [aws-]s3: If multiple S3 requests were failing concurrently, the process could have crashed.
- Fixes: Panic: Buffer write out of range
DOV-6242: "Expunged GUID mismatch" error in indexes could not be easily repaired. Such expunge requests are now ignored.
DOV-6254: fs-fscache always attempted to rmdir() the fscache root directory. This may have resulted in "Permission denied" errors being logged.
DOV-6260: imap: GETMETADATA with mailbox containing wildcards would fail with "NO [PERM]" rather than just skip the folders that cannot be accessed. The updated behavior is that if any folder is not accessible is skipped from the wildcard expansion.
DOV-6284: auth service: Request log lines often have no request prefix, making it unclear to which request the log line applies and which sub-system is involved.
DOV-6315: Prepared statements may have leaked memory if connecting to Cassandra failed.
DOV-6318: obox keeps mail files open until the transaction is committed. This results in excessive memory usage with large transactions, such as used by dsync. Regression since v2023.1.
DOV-6322: dsync may have been committing each saved mail independently instead of using dsync_commit_msgs_interval setting, which caused it to work inefficiently.
DOV-6394: Submission and LMTP services accepted termination of DATA payloads with bare CR/LFs guards instead of strictly CRLF-"."-CRLF as specified by RFC 5321.
DOV-6407: "doveadm fts check fast" didn't handle correctly fts.S cache files where some triplets contained per-folder message counts and some didn't. If there were any missing ones, the output should have been "?", but instead a wrong number was output.
DOV-6409: The last userdb's "default_fields" and "override_fields" are used, even if the user isn't found from that userdb.
DOV-6409: If passdb uses "userdb_" prefixed field, the first userdb's "default_fields" are added to the request, even if the user isn't found from the first userdb.
DOV-6410: Only the last userdb lookup was cached in auth_cache.
DOV-6451: Cassandra batch queries were hanging if they were sent while Cassandra wasn't connected to.
DOV-6480: Dovecot would send booleans as strings for auth policy requests.
DOV-6488: fs-dictmap may have assert-crashed in some situations.
- Fixes: Panic: file dict-client.c: line 826 (client_dict_wait): assertion failed: (io_loop_is_empty(dict->dict.ioloop))
DOV-6489: Some mail access patterns inside virtual-attachments folders (e.g. searching headers) may result in a crash.
DOV-6527: login-intercept didn't handle STARTTLS correctly. It was writing the encrypted network traffic into the intercept files.
DOV-6550: metacache rescanning logged unnecessary stat() errors when there were symlinks pointing to nonexistent files.
DOV-6577: login-intercept plugin wasn't working correctly with IMAP COMPRESS. The compression is now handled by the imap-login process when Dovecot proxies are used with trusted backends.
DOV-6592: In some cases the doveadm-server failed to restore properly the privileges after dropping them to serve the requests. This caused failures during next authentication phases reusing the same process.
DOV-6603: Fixed a potential hang between doveadm-to-doveadm TCP connections.
DOV-6610: Using fs-dictmap with fs-compress or fs-crypt may have caused unnecessary Cassandra lookups (although these were mostly avoided by using fs-fscache).
DOV-6611: Using fs-compress or fs-crypt with fs-dictmap may have resulted in "Object not found from dict" errors that didn't fix itself.
DOV-6622: imapc failed with "Opening mailbox failed: EXISTS not received" if the IMAP server violated the QRESYNC standard by advertising it in CAPABILITY reply, but without sending untagged "OK [CLOSED]" after SELECTing a folder (at least Zimbra). This is now fixed automatically by reconnecting. To avoid the reconnection, use imapc_features=no-qresync setting. v2.3.21 regression.
DOV-6623: imapc with delayed logins didn't automatically reconnect to server if the disconnection happened before opening a folder.
DOV-6660: "doveadm who" output didn't contain alternative usernames for hibernated IMAP sessions.
DOV-6687: If both "host" and "hostip" fields are set to IP and they differ, and end up in source proxy, infinite proxying loop can happen.
DOV-6687: If proxy field parsing fails, connection might have gotten stuck.
DOV-6702: When multiple DNS lookups were done in parallel, wrong IP address could have been returned in some cases, especially after DNS lookup errors. This mainly affected proxying if DNS lookups were being used.
DOV-6715: Fixed assert-crash if compressed/encrypted mail couldn't be written to mail_temp_dir (due to out of disk space).
- Fixes: Panic: file istream-seekable.c: line 238 (read_from_buffer): assertion failed: (*ret_r > 0)
DOV-6716: Fix assert-crash in master process after "Process .. is ignoring idle SIGINT" error.
- Fixes: Panic: file service-monitor.c: line 60 (service_kill_idle): assertion failed: (processes_to_kill <= service->process_avail)
DOV-6718: Metacache flushing (via "doveadm flush" or metacache-worker) crashed if fs-fts-cache was enabled and the flush operation started by doing dsync-based index merging.
DOV-6775: mail-intercept crashes with obox and mail_attachment_detection_options=add_flags.
- Fixes: Panic: file obox-save.c: line 215 (obox_save_update_keywords): assertion failed: (!save_mail->finished)
DOV-6776: mail-intercept plugin always opened mail bodies when copying mails.
DOV-6828: When reading encrypted files, a signal 11 crash can occur.
DOV-6919: doveadm auth test/login: Internal failures were returned with EX_NOPERM (77) instead of EX_TEMPFAIL (75) exit code. This made it difficult to distinguish between wrong password and internal failure.
DOV-6930: imapc may have caused a timeout leak in IDLE processing, resulting in a crash later. This was mainly visible sometimes when using shared folders via imapc.
DOV-6931: "doveadm save" and lib-smtp mail sending could have caused lseek() errors with certain kind of email input. This resulted in failing to save/send the mail.
DOV-7005: fixed controller API where response to Prometheus polls were stuck if "Metrics Export" feature is not enabled. Returns an empty response instead.
DOV-7006: Fixed controller's wrong parsing of Cassandra servers list if it contains spaces after the comma.

Obox Testing

AWS S3

Testing Date	2024-08
Testing Region(s)	eu-central-1
API Version(s) Tested	2006-03-01
API Documentation (at time of testing)	https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html

Azure Blob

Testing Date	2024-08
Testing Region(s)	northeurope
API Version(s) Tested	2023-08-03
API Documentation (at time of testing)	https://learn.microsoft.com/en-us/rest/api/storageservices/version-2023-08-03

Scality sproxyd

Testing Date	2024-08
API Version(s) Tested	8.5.8.6
API Documentation (at time of testing)	https://documentation.scality.com/RING/8.5.8/reference/connectors/sproxyd/index.html