Developer Summit 1

The first Scientific Python Developer Summit (May 22-26, 2023) brought together 34 developers at the eScience Institute at the University of Washington to develop shared infrastructure, documentation, tools, and recommendations for libraries in the Scientific Python ecosystem.

Pre-summit planning#

Prior to the summit we held several hour-long planning meetings:

Summit execution#

Morning group check-in

At the summit, we had a brief check-in and then split into several groups based on each developers time and interests. Raw work progress and log have been collected in a document, we highlight just a few of things we accomplished below:

Sparse arrays#

Almost a quarter of the group worked on sparse arrays for the entire week. This work is part of a larger, multi-year effort to improve and expand SciPy’s sparse array API, which will eventually involve removing the sparse matrix API and eventually np.matrix.

More details can be found in the Developer Summit 1: Sparse blog post.

Scientific Python Ecosystem Coordination documents#

We made significant progress on several SPECs, which had been drafted during previous sprints.

Snapshot of the current SPECs and their endorsements

SPEC 0—Minimum Supported Versions , an updated and expanded recommendation similar to the NEP 29, was discussed and endorsed by several core projects.

SPEC 1—Lazy Loading of Submodules and Functions was discussed and endorsed by two core projects.

SPEC 2—API Dispatch was discussed (in a follow-up video meeting just after the summit) and is in the process of being marked as withdrawn or something similar.

SPEC 3—Accessibility was discussed and updated. We hope to see it endorsed by several core projects in the near future.

SPEC 4—Using a creating nightly wheels was rewritten, a helper GitHub action upload-nightly-action was created, and PRs to update the various projects to use the new nightly wheels location were made. The updates are now complete and the SPEC was endorsed by two core projects.

We anticipate several more core projects to endorse the existing SPECs over the coming months and we are now holding regular SPEC steering committee meetings to continue developing and expanding the SPECs.

Community building#

We created a comprehensive community guide to empower projects in fostering their communities. This guide includes essential information on the role of community managers, along with practical strategies for community meetings, outreach, onboarding, and project management.

Development Documentation#

We created a development guide, a new project template, and existing project review.

Serendipitous Collaboration#

One of the fun things that happens at summits like these are the chance encounters of people from different projects. For example, a couple of attendees worked on creating a co-collaboration network across the broader scientific python ecosystem. This gave us the opportunity to look at how contributors collaborate across projects. We could see how the bigger projects were all clustered together as there are multiple contributors who share maintenance duties for multiple projects. We could also, for example, see how the Scikit-HEP cluster was a bit further away from the usual scientific Python cluster. An action item for us :) We need more collaboration!!

Visualization of co-collaboration network

Pytest plugins/Sphinx extensions#

Several attendees worked on pytest plugins and Sphinx extensions:

pytest-regex was created to support selecting tests with regular expressions.
pytest-doctestplus was moved upstream into the Scientific Python organization. The summit provided new momentum to develop new features (e.g. produce updated docstring), and to use it for the NumPy documentation testing.
sphinx-scientific-python, a new extension as a home for various features from the ecosystem, e.g., we agreed on bringing existing extensions from MNE tools to this extension.
pydata-sphinx-theme updates

SciPy release management progress#

The first release candidate of SciPy 1.11.0 was published on PyPI on May 31, 2023, five days after the conclusion of the summit. The summit facilitated high-bandwidth decision making on several proposed SciPy code changes by allowing the current SciPy release manager (Tyler Reddy, Los Alamos National Laboratory) to consult with other SciPy core developers in person. Specific code changes were discussed with the following SciPy maintainers: Stefan van der Walt (scipy.ndimage), CJ Carey (scipy.sparse), Matt Haberland (scipy.stats), and Pamphile Roy (scipy.stats). When SciPy releases are performed out of band from the summit, the release manager will often have to delay incorporation of useful code changes to the next release six months later, due to lack of availability of the pertinent domain experts.

Package metrics#

We factored out a general developer statistics package from our prototype developer statistics website.