Maintainers information

Packages upgrading

This section contains information about how and when to upgrade the packages used by GEMSEO and by the development environments.


GEMSEO is a library and not a self contained application, it can be installed in environments with varying and unknown constraints on the versions of its dependencies. Thus the versions of its dependencies cannot be pinneds, but a range of compatible versions shall be defined.

All the dependencies shall be defined in setup.cfg, this files does not tell where the packages will be pulled from. The dependencies could be provided by the packages repositories pypi, anaconda or conda-forge.

Getting GEMSEO to work with a common set of packages versions on several platforms and python versions is tricky and challenging. This kind of work is mostly done by trials and errors.

To reduce maintenance and complexity, our testing environments shall have the same packages providers for all the platforms and all the python versions. Furthermore it shall be identical to the references end-user environments under the same constraints.

In the context of tox, the versions of the dependencies that shall be installed with conda are defined in requirements/gemseo-conda-python2,3.txt, they are pulled from conda-forge. All other dependencies are installed with pip, they are pulled from pypi.

When a dependency is changed, setup.cfg shall always be modified. If the changed dependency is installed with conda, then both gemseo-conda-python2,3.txt and environment-py2,3.yml shall be modified.

Documentation files like CREDITS.rst and dependencies.rst shall also be updated accordingly.

CI cache

To optimize the usage of the CI cache with gitlab, the cache shall match closely the contents of the tox environments used for testing. This is currently done by defining a cache affinity with the dependencies of the environments and the environment name. The dependencies are stored in the file requirements/ which is created automatically by a custom pre-commit hook defined in :py:`tools/`.

Development dependencies

As opposed to the dependencies of GEMSEO, the development dependencies can be fully controlled. Thus their versions are pinned so all developers are provided with reproducible and working environments. The dependencies shall be updated at least once in a while (couple months) to benefit from packages improvements and bug fixes.

This is done with pip-tools and from input requirements files. These input requirements files contain the minimum pinning requirements and are intended to be modified by maintainers. The pip-tools package provides the pip-compile which can process an input requirements file to produce a fully pinned requirements file.

We have the following input requirements files:

  • with development tools with python 3 only,

  • for building the documentation with python 3 only.

To update them:

conda run -p .tox/dev pip-compile -U requirements/
conda run -p .tox/doc pip-compile -U requirements/


Append -win to the environment names under windows.


To reduce discrepancy among the environments, requirements/test-python3.txt shall be working for all the python 3 testing environments.

Git hooks are defined and run with pre-commit. It relies on packages that are managed with pre-commit instead of pip-tools. To update them:

conda run -p .tox/dev pre-commit autoupdate


Append -win to the environment names under windows.


All environments and tools shall be checked whenever dependencies have been changed.

Test dependencies

The test dependencies are defined in setup.cfg so a end-user can easily run the GEMSEO tests.

To update them, change the test key of the [options.extras_require] section in setup.cfg, then execute

tox -e style

This will call a pre-commit hook that will update requirements/ Then update the actual test requirements with:

conda run -p .tox/dev pip-compile -U requirements/ -o requirements/test-python3.txt
conda run -p .tox/py27 pip-compile -U requirements/ -o requirements/test-python2.txt


Append -win to the environment names under windows.


All environments and tools shall be checked whenever dependencies have been changed.

Anaconda testing

Before a release or whenever required, the package for conda-forge and the anaconda environment files shall be tested.

Anaconda package

Run (append -win on windows)

tox -e anaconda-package

Anaconda environment

Run (append -win on windows)

tox -e anaconda-envs

Pypi distribution testing

Run (append -win on windows)

tox -e py27-pypi,py36-pypi,py37-pypi,py38-pypi

Making a new release

  1. Create a release branch.

  2. Make sure the full test suite passes.

  3. Replace Unreleased by the new version in CHANGELOG.rst.

  4. Hardcode the version number in

  5. Push the branch.

  6. Build the docs for this branch on rtd, check the version and changelog.

  7. Merge to master.

  8. Tag.

  9. Run tox -e create-dist to create the distribution archives.

  10. Run twine upload dist/* -u <your login> to upload to

  11. Run tox -e py*-pypi{-win} for all envs to test the pypi package.

  12. Update the recipe for conda-forge once the update bot sends the PR.

  13. Test the conda-forge packages (TODO: create a tox target)

  14. Merge master to develop so the last tag is a parent commit for defining the dev versions.