Kubernetes Deployments

New Features

• DOV-6950: controller: It's possible to configure the Service type of Controller API via controller.api.service.type parameter (default to the current/former value: ClusterIP).

Changes

• DOV-7027: controller: It's no longer needed to mention the "maxmemory-policy" setting in the redis.commonConfiguration (since it's configured under the hood elsewhere).
• DOV-7028: controller: It's now possible to configure any registry in both global.image.registry and image.registry parameters.

Dovecot Cluster Controller

New Features

• DOV-6744: Cluster controller now has the possibility to run in dry-run mode by setting the key controller.worker.dryrun to true. This mode will log actions like changing backend status online or offline, or when group would move.
• DOV-6747: Cluster controller tracks group moves for their progress. If it seems like the move is very slow or stuck it will force the move directly by updating GeoDB. Two new settings are introduced: GROUP_MOVE_START_TIMEOUT_SECS defines how long a group can be in the "move started" stage before it's forced and GROUP_MOVE_FINISH_TIMEOUT_SECS defines how long a move can take before it's forced. The following conditions trigger the force move:
  • If controller marked a group to be moved to another backend but dovecot backends didn't pick up the move or update GeoDB within the threshold period.
  • If move is in progress but it will take more than GROUP_MOVE_FINISH_TIMEOUT_SECS seconds for it to finish.
  • If there has been a long pause in group move status update by the backend.
• DOV-7078: Controller adds support for configuring Cassandra load balancing Policy. The two options are "RoundRobin" and "DCAwareRoundRobin". The latter prefers local cluster nodes and uses remote nodes as last resort.

Changes

• DOV-6709: Cluster controller is now configured to have unlimited connection retries (with exponential back-off) in case of Redis connection errors or if Redis reports out of memory.
• DOV-6743: Cluster controller now checks before evacuating backends that have high failure rate in health monitoring and if more than 75% of all backends need to be evacuated. In this situation, no group move is done. Instead a critical error is logged and the administrator is expected to manually check the root cause for the severe issue.
• DOV-7057: Refactored Cassandra client library. This change makes cluster controller compatible with ScyllaDB.
• DOV-7064: Controller configuration option CASSANDRA_KEYSPACE is renamed to CASSANDRA_GEODB_KEYSPACE.
• DOV-7099: Controller configuration option CASSANDRA_SITE is renamed to CASSANDRA_LOCAL_DC.
• DOV-7198: Non-essential packages were removed from the image
• DOV-7198: Non-essential utilities were removed from the image

Improvements

• DOV-7155: Controller now warns if Cassandra queries take more than 1 second. The warnings are gated so that no same warning will be logged within 5 seconds.
• DOV-7168: Speed optimizations for Cassandra queries by doing data filtering on the client side instead of repeatedly using ALLOW FILTERING.
• DOV-7210: Optimised backend and group statistics gathering and processing. This fixes an issue where Prometheus metrics scraping failed due to cached metrics being too old.

Fixed Issues

• DOV-6870: Add workaround for memory leak in controller worker process due to a bug in underlying celery library.
• DOV-6967: fixed controller API where response to Prometheus polls were stuck if "Metrics Export" feature is not enabled. Returns an empty response instead.
• DOV-6986: Fixed cluster controller API to not accept IP addresses or valid UUID strings as host name when creating a backend.
• DOV-7001: Fixed controller's wrong parsing of Cassandra servers list if it contains spaces after the comma.
• DOV-7119: Fix wrong configuration read for minimum samples needed in load balancing.

Dovecot

New Features

• DOV-6805: Support for Azure blob batch deletion

Changes

• DOV-6910: Docker compose now runs all components except redis as non-root user without extra privileges. Containers are marked read-only with tmpfs for state data.
• DOV-7068: dovecot.service: Added dependency for time-sync.target. This way Dovecot won't be started before time has been synchronized. systemd-time-wait-sync.service also needs to be enabled.
• DOV-7153: fts-kuromoji: Add U+30FB as tokenizer delimiter.
• DOV-7225: Dovecot now requires at least OpenSSL v1.1.1 to work.

Removed Items

• DOV-6945: cassandra: Removed ssl_verify=cert-dns. This behavior is vulnerable to MITM attacks, so it's not safe to use.

Improvements

• DOV-6792: intercept: Added login and logout events for Utimaco X3.
• DOV-7007: Discrete group_by in stats now allows modifiers via %variables.
• DOV-7015: Add proxy_no_multiplex passdb extra field to disable multiplex iostreams in proxies towards the next hop. This is required when doing IMAP proxying with Dovecot v3.0 proxy - v2.3 proxy - v3.0 backend.
• DOV-7026: Added --oldest-safe-timestamp option to doveadm metacache status.

Fixed Issues

• DOV-6406: imapc: When a shared mailbox's INBOX is implicitly shared due to a submailbox being shared, the \Noselect flag of INBOX was not respected.
• DOV-6922: doveadm who crashed when using the json formatter.
• DOV-6997: The %u format specifier in pop3_logout_format was always empty.
• DOV-7031: Failing to do FTS indexing for a virtual folder may have crashed instead of simply failing with an error.
  • Fixes: Panic: file fts-backend-dovecot.c: line 354 (fts_backend_dovecot_flush_expunged): assertion failed: (last_indexed_uid <= next_indexed_uid)
• DOV-7050: Dovecot may have wrongly logged "Time moved forwards" warnings. This mainly happened on idling servers.
• DOV-7082: ssl_ja3_hash did not work in a blocking passdb.
• DOV-7111: Sieve script would crash if more than one :optional script was missing.
• DOV-7117: If IMAP client disconnected during IDLE unhibernation, the session may have become stuck for 30 minutes or crashed immediately.
• DOV-7120: When dsync merges folder trees, it may crash.
  • Fixes: Panic: file dsync-mailbox-tree-sync.c: line 575 (node_mailbox_trees_cmp): assertion failed: (ret != 0)
• DOV-7121: fs-dictmap: If delete-dangling-links option auto-deleted a lost self/diff index bundle for a folder that wasn't in folder list index (due to corruption or lost user index bundle), the folder could have been lost entirely.
• DOV-7154: Fixed various hangs and a crash if IMAP client sent command lines longer than imap_max_line_length (default: 64 kB).
  • Fixes: Panic: file imap-client.c: line 1098 (client_check_command_hangs): assertion failed: (!have_wait_unfinished || unfinished_count > 0)
• DOV-7206: If mdbox resyncing failed early, it crashed. This mainly happened due to a locking timeout caused by another process that was doing a long-running resync.
• DOV-7209: If auth caching is enabled, and both passdb and userdb return changed username, then an initial login (e.g. imap) followed by a userdb lookup (e.g. lmtp) returned the unchanged username.
• DOV-7276: doveadm auth command may crash if aborted with a signal at the same time as when the auth finishes.
• DOV-7278: fs-s3: Bulk deletes didn't contain the maximum number of bulk delete requests when used with fs-dictmap.