Maintainers information

Packages upgrading

This section contains information about how and when to upgrade the packages used by GEMSEO and by the development environments.

Dependencies

GEMSEO is a library and not a self contained application, it can be installed in environments with varying and unknown constraints on the versions of its dependencies. Thus the versions of its dependencies cannot be pinned, but a range of compatible versions shall be defined.

All the dependencies of GEMSEO are defined in setup.cfg, this files does not tell where the packages will be pulled from. The dependencies could be provided by the packages repositories pypi, anaconda or conda-forge.

Getting GEMSEO to work with a set of packages versions common to several platforms and python versions is tricky and challenging. This kind of work is mostly done by trials and errors.

In addition to the dependencies of GEMSEO, setup.cfg also defines optional dependencies used for running the tests or building the documentation. These are defined in the [options.extras_require] section.

Dependencies for development

The dependencies used for development shall be fully controlled so all developers are provided with reproducible and working environments. The dependencies shall be updated at least once in a while (couple months) to benefit from packages improvements, security and bug fixes.

The dependencies update is done with pip-tools and eventually from input requirements files. These input requirements files contain the minimum pinning requirements and are intended to be modified by maintainers. The pip-tools package provides the pip-compile which can process an input requirements file to produce a fully pinned requirements file. The actual call to pip-tools is done via tox (see below).

Warning

All environments and tools shall be checked whenever dependencies have been changed.

Documentation files like CREDITS.rst and dependencies.rst shall also be updated accordingly.

Whenever a dependency defined in setup.cfg is changed, update the requirements for the testing and doc environments of tox:

tox -e update-deps-test-py37,update-deps-test-py38,update-deps-test-py39,update-deps-test-py310,update-deps-doc

The dependencies for the check and dist environments of tox are defined in:

  • check.in: for checking the source files.

  • dist.in: for creating the distribution.

Update the requirements for the those environments of tox:

tox -e update-deps-check
tox -e update-deps-dist

Testing pypi packages

Run

tox -e pypi-pyX

for all the supported Python versions X, e.g. tox -e pypi-py39.

Testing conda-forge packages

Run

tox -e conda-forge-pyX

for all the supported Python versions X, e.g. tox -e conda-forge-py39.

Updating the changelog

To avoid rebase and merge conflicts, the changelog is not directly updated in a branch but updated once a release is ready from changelog fragments. Changelog fragment is a file that contains the part of the changelog of a branch, named with <issue number>.<change kind>.rst and stored under changelog/fragments. The update is done with towncrier:

towncrier build --version <version number>

Making a new release

  1. Create a release branch.

  2. Make sure the full test suite passes.

  3. Update the changelog.

  4. Push the branch.

  5. Build the docs for this branch on rtd, check the version and changelog.

  6. Merge to master.

  7. Tag.

  8. Run tox -e dist to create the distribution archives.

  9. Run twine upload dist/* -u <your login> to upload to pypi.org.

  10. Test the pypi packages.

  11. Update the recipe for conda-forge once the update bot sends the PR.

  12. Test the conda-forge packages.

  13. Merge master to develop so the last tag is a parent commit for defining the dev versions.

  14. Push develop.

Making a new release for plugins

  1. Create a release branch.

  2. Update the required gemseo version in setup.cfg.

  3. Update the environments dependencies (update-deps) while setting the environment variable GEMSEO_PIP_REQ_SPEC="gemseo".

  4. Update the changelog.

  5. Push the branch.

  6. Make sure the full test suite passes.

  7. Merge to master.

  8. Tag.

  9. Run tox -e dist to create the distribution archives.

  10. Run twine upload dist/* -u <your login> to upload to pypi.org.

  11. Test the pypi packages.

  12. Update the recipe for conda-forge once the update bot sends the PR.

  13. Test the conda-forge packages.

  14. Merge master to develop so the last tag is a parent commit for defining the dev versions.

  15. Update the environments dependencies (update-deps) without setting the environment variable GEMSEO_PIP_REQ_SPEC.

  16. Push develop.

Mirroring to github

To mirror a project from gitlab to github:

  • Clone the repository on github,

  • Enable push mirroring on the gitlab repository setting page.