Software Architecture 2020

This migration primarily entails a general update of the available software on our servers. It involves a move of the account to new servers - running with the 2020 architecture. All servers (HTTP, SSH, databases, etc) are likely to change.

Why should there be migration?

The main purpose of architecture migrations is to update our servers to a newer version of their operating system (Debian). It is important to proceed with these updates for two reasons:

• Security: Older software versions are not supported forever by their authors or maintainers. When such software ceases to be supported, it continues to run, but new vulnerabilities are no longer fixed. Continuing to use these old versions would therefore create a serious safety and reliability risk for your applications and data.

• Be able to use recent software versions: The older an operating system is, the more likely it is to have issues in running recent software. In order to continue to provide you with the latest versions of software that we offer (or that you can install yourself in your account), it is important to be on a fairly recent version of the operating system.

These infrastructure migrations are performed approximately every four years. This is roughly the lifetime of a Debian version (five years, but you should allow for the incompressible amount of internal development time before switching to the newer version). This seems to be a fair balance, avoiding too frequent migrations, but allowing you to run almost all recent software releases.

This document presents the main incompatibilities introduced by this migration. We have tried to be as comprehensive as possible, but it is quite difficult to be absolutely exhaustive.

Apache

Apache 2.2 will no longer be available. Accounts using this version will automatically switch to Apache 2.4.

Global directives added by our users, either in Web > Configuration, or in the configuration of a Custom Apache site, will now be inserted after the default directives. This now allows our users to overwrite these default directives.

Databases

MySQL

MariaDB will be updated to version 10.4.

PostgreSQL

PostgreSQL will be updated to version 12.

MongoDB

MongoDB will be updated to version 4.2.

CouchDB

CouchDB will be updated to version 3.0.

Languages

PHP

• Minor removed versions will automatically be replaced by the closest minor version (e.g. 7.4.4 for 7.4).
• The bcmath, calendar, exif, ftp, soap, xmlreader, xmlrpc and zip extensions are now loaded automatically. You can remove explicit loading directives from your custom php.ini if required.

Python

• Minor removed versions will automatically be replaced by the closest minor version (e.g. 3.8.2 for 3.8). If you created any virtualenvs with these versions, you will need to recreate them.

When changing the minor, rather than recreating the virtual environment, you can use the following command:

python -m venv --upgrade [myenv]/

Ruby

• Minor removed versions will automatically be replaced by the closest minor version (e.g. 2.6.5 for 2.6).

Node.js

• Minor removed versions will automatically be replaced by the closest minor version (e.g. 12.16.1 for 12). `

Elixir

• Minor removed versions will automatically be replaced the closest minor version.

Java

Java will soon become a major language, manageable via the administration panel.

• The versions now available are 8.0.41, 11.0.28 and 14.0.36. Versions 7, 9 and 10 will be removed.

• The default version is now 11. This is the version run when using the java binary.

• Until now, to use a particular version of Java, you could fetch the binary from /usr/lib/jvm. This is no longer possible: instead, you can enforce a different version by setting the JAVA_VERSION environment variable. For example, to run version 8: JAVA_VERSION=8 java.

Miscellaneous

• TLS 1.0 and 1.1 are now disabled by default for HTTP Protocol, as these older protocols have security holes. You can re-enable them by going to Web > Configuration > SSL and then choosing the Old configuration.

• The temporary files directory (TEMPDIR) is now ~/admin/tmp instead of /tmp. PHP sessions, for example, are created in this directory.

• For sites such as Node.js, Elixir, and User Program, the internal IP (defined in the IP environment variable) that your application should listen on will change to IPv6. The new IP will be given in the helptext of the website command.

• The ALWAYSDATA_HTTPD_PORT and ALWAYSDATA_HTTPD_IP environment variables are no longer available, you must use PORT and IP.

• The PATH environment variable now includes the local paths for all languages, e.g. ~/.local/bin, ~/npm-packages/bin, etc., even in non-login or non-interactive SSH connections, and in your HTTP applications.

• The own user of your $HOME directory (e.g. /home/foobar, if your account is foobar), previously set to your username (e.g. foobar), now becomes root. The owner group will still match your username (here, foobar), so it won’t change anything in practice. Your home directory permissions will therefore be reset to 0770.

• Language binaries are now installed on-demand. In case your would need, for instance, /usr/alwaysdata/python/2.7.18/lib/libpython2.7.so, make sure you have launched python in version 2.7.18 at least once.

Miscellaneous Updates

A lot of software and libraries will be updated (our servers are running Debian Buster, formerly Debian Jessie). Among the notable updates :

• Erlang 21.2 (previously 19.2)
• GDAL 2.4 (previously 1.10)
• git 2.20 (previously 2.19)
• Go 1.11 (previously 1.7)
• MapServer 7.4 (previously 7.0)
• Mercurial 4.8 (previously 3.9)
• PROJ 5.2 (previously 4.8)
• QGIS 3.4 (previously QGIS 2.4)

VPS/Dedicated Environment

The following services, when installed, will be updated:

• RabbitMQ, version 3.7.8.
• Redis, version 5.0

Upgrades of MySQL (not MariaDB) and ElasticSearch will be discussed with users on a case-by-case basis.

Only the language versions explicitly used, either in the Web > Sites section or in the Environment section, will now be preinstalled on the system. For example, if neither the default version of Python (defined in Environment > Python) nor any of your sites (Web > Sites) uses Python 2.4.6, then this version will no longer be preinstalled. However, it will be automatically installed if you create a site with this version of Python, or if you change the default version of Python.

Migration preparation

A number of actions can be performed on the 2017 architecture:

• switch to Apache 2.4 in the Web > Configuration > Apache menu,

• change the TLS configuration to Intermediate in the Web > Configuration > SSL menu,

• replace ALWAYSDATA_HTTPD_PORT and ALWAYSDATA_HTTPD_IP environment variables in your applications with PORT and IP,

• change the language versions to the last minor. This is done in the Environment menu and/or at the sites level in Web > Sites. You can, for example, switch to PHP 7.3.9 to replace PHP 7.3.0.

We encourage you to make these changes in production, before you migrate.

Databases

In parallel to the Buster migration, we provide database migrations. You can test them via the Test button. All your databases and database users are copied to a temporary server, running the new versions.

Users on shared servers can perform these tests in parallel to the Buster migration. The Buster migration will close the other migrations.

For users of VPS and dedicated servers, the database migration test is associated to the Buster migration. The Test button will only copy the databases to the test server. Regarding HTTP deployment refer to the Migration preparation section.

Migration process

When you click on the Migrate button the process usually starts immediately, but there may be a delay of several minutes depending on the number of users migrating at the same time. The migration is done in several successive steps, service by service. For example, your files will be migrated before your databases.

• The migration of your files, performed first, will cause a downtime of your websites (which will display an internal error), your scheduled tasks, or your remote accesses (SSH, FTP, etc.). However, the downtime will be short (a few seconds in general, more if you have tens of thousands of files), as these files are pre-copied beforehand.

• During database migration, connection to databases is interrupted. On average, there will be a minute of downtime per Gb of data. It may be judicious to set up a static maintenance page on your websites to avoid a generic database connection error.

You can check if the migration is complete via the Tasks menu (top right corner of your administration interface).

Tips & common issues

• Accounts are relocated during the migration. There may be a DNS propagation delay and it is necessary to update its known_hosts file for the SSH connection. You can do this via the command ([account] to be replaced by the account name):

$ ssh-keygen -R ssh-[account].alwaysdata.net

• Drupal

  • CSS and JS files precompression in versions prior to 8.9.

• Segmentation fault issues with psycopg2.