Maintainers information

Packages upgrading

This section contains information about how and when to upgrade the packages used by GEMSEO and by the development environments.


GEMSEO is a library and not a self contained application, it can be installed in environments with varying and unknown constraints on the versions of its dependencies. Thus the versions of its dependencies cannot be pinneds, but a range of compatible versions shall be defined.

All the dependencies shall be defined in setup.cfg, this files does not tell where the packages will be pulled from. The dependencies could be provided by the packages repositories pypi, anaconda or conda-forge.

Getting GEMSEO to work with a common set of packages versions on several platforms and python versions is tricky and challenging. This kind of work is mostly done by trials and errors.

To reduce maintenance and complexity, our testing environments shall have the same packages providers for all the platforms and all the python versions. Furthermore it shall be identical to the references end-user environments under the same constraints.

In the context of tox, the versions of the dependencies that shall be installed with conda are defined in requirements/gemseo-conda-python2,3.txt, they are pulled from conda-forge. All other dependencies are installed with pip, they are pulled from pypi.

When a dependency is changed, setup.cfg shall always be modified. If the changed dependency is installed with conda, then both gemseo-conda-python2,3.txt and environment-py2,3.yml shall be modified.

Documentation files like CREDITS.rst and dependencies.rst shall also be updated accordingly.

CI cache

To optimize the usage of the CI cache with gitlab, the cache shall match closely the contents of the tox environments used for testing. This is currently done by defining a cache affinity with the dependencies of the environments and the environment name. The dependencies are stored in the file requirements/ which is created automatically by a custom pre-commit hook defined in :py:`tools/`.

Development dependencies

As opposed to the dependencies of GEMSEO, the development dependencies can be fully controlled. Thus their versions are pinned so all developers are provided with reproducible and working environments. The dependencies shall be updated at least once in a while (couple months) to benefit from packages improvements and bug fixes.

This is done with pip-tools and from input requirements files. These input requirements files contain the minimum pinning requirements and are intended to be modified by maintainers. The pip-tools package provides the pip-compile which can process an input requirements file to produce a fully pinned requirements file.

We have the following input requirements files:

  • for running the tests with python 2 and 3 (this file is automatically generated by a pre-commit hook),

  • with development tools with python 3 only,

  • for building the documentation with python 3 only.

To update them:

conda activate .tox/dev
pip-compile -U requirements/ -o requirements/test-python3.txt
pip-compile -U requirements/
conda activate .tox/doc
pip-compile -U requirements/
conda activate .tox/py27
pip-compile -U requirements/ -o requirements/test-python2.txt


To reduce discrepancy among the environments, requirements/test-python3.txt shall be working for all the python 3 testing environments.

Git hooks are defined and run with pre-commit. It relies on packages that are managed with pre-commit instead of pip-tools. To update them:

conda activate .tox/dev
pre-commit autoupdate


All environments and tools shall be checked whenever the dependencies have been changed.

Anaconda testing

Before a release or whenever required, the package for conda-forge and the anaconda environment files shall be tested.

Anaconda package

Run (append -win on windows)

tox -e anaconda-package

Anaconda environment

Run (append -win on windows)

tox -e anaconda-envs

Pypi distribution testing

Run (append -win on windows)

tox -e py27-pypi,py36-pypi,py37-pypi,py38-pypi

Full packaging

The full package is an archive named gemseo-full-package.tbz for delivering a release, it contains:

  • GEMSEO distribution

  • generated documentation

  • tests

  • anaconda environment files

  • README.rst

Create it with

tox -e full-package