===================
Upgrading Mailman 3
===================

This includes the requirements to upgrade to Mailman 3 suite, which includes:

- `Mailman Core <https://pypi.org/project/mailman/>`_ 3.x
- `Postorius <https://pypi.org/project/postorius/>`_ 1.x
- `MailmanClient <https://pypi.org/project/mailmanclient/>`_ 3.x
- `Django-mailman3 <https://pypi.org/project/django-mailman3/>`_ 1.x
- `Hyperkitty <https://pypi.org/project/Hyperkitty/>`_ 1.x

If you are interested in the full change log for each component, please refer to
documentation of each project.

Requirements
============

Python 3
--------

Postorius, Django-mailman3 and Hyperkitty have been ported to Python 3
now. Generally the last few versions of Python and Django are supported but the
supported versions keep on changing over time. Release announcements should
have information about which vesions of Python are supported.

Migration Steps
===============

Distro packages
---------------

If you are using distro packages, they should handle the upgrade process
themselves. You *may* need to perform some changes after the upgrade is done,
please check with the distro package's documentation for those instructions.

Docker images
-------------

If you are using `container images
<https://github.com/maxking/docker-mailman>`_ please refer to their
documentation or release notes on steps to upgrade a new version.

Virtualenv Install
------------------

If you are following :ref:`virtualenv installation guide from
here<virtualenv-install>`, these are the steps to upgrade to a newer version of
Mailman Core.

Pre-upgrade
************

First, you need to stop the running Mailman services before the upgrade to make
sure that you don't endup in a bad state. You can stop the two systemd
services, for Core and Mailman-Web::

  $ sudo systemctl stop mailman3
  $ sudo systemctl stop mailmanweb

Then you need to switch to ``mailman`` user, activate the virtualenv::

  $ sudo su mailman
  $ source /opt/mailman/venv/bin/activate

Upgrade package
****************

Finally, upgrade your packages::

  (venv) $ pip install -U mailman postorius django-mailman3 hyperkitty mailman-web


.. warning:: It is important to upgrade all the packages together since they
             depend on each other for critical functionality. There are no
             compatability guarantees with a new version of one package and an
             old version of another package.

             We often update the minimum version required for a dependency in
             Packages' metadata but that doesn't always result in new versions
             being installed. There is also no direct dependency between django
             packages and mailman package, but they still expect the latest
             version of Mailman core.

Post upgrade
************

Post upgrade, you need to do a bunch of other tasks. First would be to make
sure that your database schema is updated to handle the new versions::


  (venv) $ mailman-web migrate

Then, make sure that all the static files, like stylesheets, javascript, images
etc are copied to a single place. This is required to have the CSS and
Javascript displayed correctly::

  (venv) $ mailman-web compress
  (venv) $ mailman-web collectstatic


Update your i18n translation caches to make sure that your installation can
pickup any new changes to strings and their translations::

  (venv) $ mailman-web compilemessages

Finally, you can re-start the services to bring up the Web Interface and
Mailman Core::

  $ sudo systemctl start mailman3
  $ sudo systemctl start mailmanweb


Upgrade to 3.2.0
================

These are specific instructions to upgrade to Mailman Core 3.2.0 or 1.2.0 for
mailman-web components (like, Postorius, Hyperkitty) from an older version.

Configuration
-------------

Postorius added a new *required* configuration flag
:setting:`POSTORIUS_TEMPLATE_BASE_URL` which should be set to URL that
Postorius is expected to be listening at. You should set it to whatever URL
your WSGI server is listening at.


Extra Migration Steps
---------------------

- Full text index for Hyperkitty needs to be re-built since our indexing engine
  doesn't maintain compatibility between index created in Python 2 and Python 3.

  Simplest way to do this to run ``python manage.py rebuild_index`` in your
  Django project. Note that if your project has a huge number of lists, this
  will take a lot of time.

  A new command ``python manage.py update_index_one_list <listname@example.com>``
  was added to Hyperkitty so that you can rebuild your index one-by-one, if you
  prefer that.


Upgrade from <3.3.1 to >=3.3.1
==============================

Mailman core 3.3.1 is the first release that actually processes bounces.  Prior
to this release, bounces were recorded in the ``bounceevent`` table in the
database, but not processed.  This leads to a situation where upgrading to
core >=3.3.1 can cause many old, stale bounces to be processed resulting in
disabling delivery to some list members who are not currently bouncing.

There are two ways to avoid this when upgrading.  The preferred way is to
enable `VERP probes
<https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/model/docs/bounce.html#verp-probes>`_.
Instead of immediately suspending delivery when a member's bounce score reaches
threshold, a probe message is sent to the member and delivery is suspended only
if the probe bounces.  This is done by putting::

    [mta]
    verp_probes: yes

in ``mailman.cfg`` before upgrading.

The other way to avoid this is to set the ``processed`` flag on all the rows
of the ``bounceevent`` table before upgrading with a query like::

    UPDATE bounceevent SET processed = 1;