Obox Storage Support: NFS

INFO

It is recommended to use @fs_defaults = nfs to configure this provider. Using this setting will set these settings by default:

Click to show settings

  fs_compress_write_method = zstd
  obox {
    fs fscache {
      path = /var/cache/mails/%{user | sha1 % 4}
      log_path = /var/cache/mails-%{user | sha1 % 4}.log
      size = 512 M
    }
    fs compress {
    }
    fs crypt {
    }
    fs posix {
    }
  }
  metacache {
    fs compress {
    }
    fs crypt {
    }
    fs posix {
    }
  }

If you are using Dovecot FTS Plugin you should also set @fts_fs_defaults = nfs, which expands to:

Click to show settings

  fts dovecot {
    fs fts-cache {
    }
    fs fscache {
      path = /var/cache/fts/%{user | sha1 % 4}
      log_path = /var/cache/fts-%{user | sha1 % 4}.log
      size = 512 M
    }
    fs compress {
    }
    fs crypt {
    }
    fs posix {
    }
  }

Prerequisites

To use Obox over NFS you need at least NFSv3 capable storage shared to all your backends.

All your hosts must be using clock syncronization. Not using clock synchronization will cause problems.

Mail delivery must be set to use LMTP for deliveries.

Setting up

You need to provision NFS shares, either all in one, or one for mail, and one for indexes. FTS indexes can be stored in same place as indexes. Recommendation is to have FTS and index shares on fast disk and mail storage can be on slower disk. If all the disks are fast, you can also have all on same share.

Configuration

Assumptions:

• /var/cache/mails filesystem is either tmpfs or a fast local disk
• /var/cache/fts filesystem is either tmpfs or a fast local disk
• /data is the NFS mountpoint

obox {
  fs fscache {
    size = 512M
    # Split users into 4 fscache directories
    path = /var/cache/mails/%{user | sha1 % 4}
    log_path = /var/cache/mails-%{user | sha1 % 4}.log
  }
  fs compress {
  }
  fs posix {
    # Use 256 hash directories for users stored in NFS
    prefix = /data/mails/%{user | sha1 % 256 | hex(2)}/%{user}/
  }
}
metacache {
  fs compress {
  }
  fs posix {
    # Use 256 hash directories for users stored in NFS
    prefix = /data/index/%{user | sha1 % 256 | hex(2)}/%{user}/
  }
}
fts dovecot {
  fs fts-cache {
  }
  fs fscache {
    size = 512M
    # Split users into 4 fscache directories
    path = /var/cache/fts/%{user | sha1 % 4}
    log_path = /var/cache/fts-%{user | sha1 % 4}.log
  }
  fs compress {
  }
  fs posix {
    # Use 256 hash directories for users stored in NFS
    prefix = /data/fts/%{user | sha1 % 256 | hex(2)}/%{user}/fts/
  }
}

It is recommended to use encryption and compression with obox and fts indexes for security and performance reasons. See Obox Configuration.

NFS tuning

NFS has many tunings, please consult your NFS vendor for details. For Dovecot, these are strongly recommended:

• actimeo=60: This or the more specific settings can be used to control NFS caching. Increasing this can reduce NFS traffic. It should be at least 60 seconds.
• nordirplus: Disable readdirplus operations, which aren’t needed by Dovecot. They can also slow down some NFS servers.
• noatime: Disable updating atime. Dovecot doesn’t need this and it may slow down NFS servers.
• root_squash: Dovecot doesn’t care about this. Typically Dovecot doesn’t store any root-owned files in NFS.
• nolock / local_lock=all: Dovecot obox format doesn't care about this. There are no locks used on NFS server side.