Packaging Working Group
The Packaging Working Group is a volunteer work group of the Python Software Foundation.
Agenda
The purpose of this working group is to support the larger efforts of improving and maintaining the packaging ecosystem in Python through fundraising (including a sponsorship program) and disbursement of raised funds. It largely focuses on efforts such as PyPI, pip, packaging.python.org, setuptools, and cross-project efforts.
Resources
- Discussion: Slack and mailing list. The archives are set to private since there is voting.
Accounting: We rely on the PSF's donation and accounting mechanisms to raise funds and disburse them to the selected recipients.
Project ideas: Fundable packaging improvements
Governance
Decisions on what fundraising and projects/efforts to support will be done by a simple majority and in the case of a tie, will escalate to the PSF Board. See the PSF Packaging WG Charter.
Administration and Contact
Donald Stufft <donald@python.org> (chair)
Dustin Ingram <di@python.org> (co-chair)
Nicole Harris <n.harris@kabucreative.com> (co-chair)
To contact the Packaging WG, email <packaging-wg@python.org>.
Members
- Alyssa Coghlan
- Ee Durbin
- Thea Flowers
- Nathaniel J. Smith
- Tzu-ping Chung
- Jannis Leidel
- (others will be added as they accept their invitation to the WG)
Meetings
As needed.
Current Projects
Fundraising
The Packaging Working Group is seeking sponsorships and grants to raise funds for fundable packaging improvements.
Sprints
We run PackagingSprints at conventions and as standalone events. We're open to companies and organizations hosting sprints and work weeks to help us move packaging forward; get in contact with a Working Group member.
PyPI Malware Reporting and Response project
The PSF has received funding from the Center for Security and Emerging Technology (CSET) to develop and improve the infrastructure for malware reporting and response on PyPI.
Summary: Develop an API that allows malware reporting and define the criteria for automated consensus based takedown and soft-deletes of packages
Schedule: One year
Meetings and Updates: See below.
Meetings and status updates:
Meeting/update
Type
Date
Past projects
PyPI Organization Account
The Python Software Foundation, with the Packaging WG's approval, funded a project to deploy organization account features in PyPI.
Summary: Organization accounts in PyPI will allow organizations to create accounts, manage users, manage projects and set permission levels for a team.
Schedule: 16 weeks that commenced on April 1, 2022
High level Roadmap: PyPI Organization Account High-level Roadmap
Detailed Roadmap: PyPI Organization Account Detailed Roadmap
Code and discussion: GitHub repository for Warehouse, and Discourse forum
Project Board: PyPI Organization Account Project Board
Deployment: pypi.org.
Manager: Shamika Mohanan
Meetings and Updates: See below.
Meetings and status updates:
Meeting/update
Type
Date
Warehouse: Facebook gift
The Packaging Working Group applied for and is receiving a gift from Facebook to implement & deploy security features for Warehouse (PyPI's codebase).
Summary: Cryptographic signing of artifacts, and malware detection. See announcement blog post, and the milestone description on GitHub.
Schedule: As of 2 January 2020, the PSF has hired contractors to carry out this work, and has commenced work.
Roadmap: WarehouseRoadmap
Code and discussion: GitHub repository for Warehouse, Zulip livechat, and Discourse forum.
Deployment: pypi.org.
Testing: To be determined
Manager: Ee Durbin
Meetings and Updates: See below.
Meetings and status updates:
Meeting/update
Type
Date
Python Package Index - Python Software Foundation's TUF key generation and signing ceremonies
Live video stream of ceremony
October 30th, 2020
Kickoff - 2019 Q4 RFP Milestone 2 - Automated Detection of Malicious Uploads
Meeting notes
December 11th, 2019
Dependency resolver and user experience improvements for pip
The Packaging Working Group applied for and is receiving funding to work in 2020 on the design, implementation, and rollout of pip's next-generation dependency resolver. The donors funding this work are the Chan Zuckerberg Initiative (USD200,000ドル) and Mozilla Open Source Support (USD207,000ドル).
Summary: Complete pip's next-generation dependency resolver, and do user experience research and design to improve pip's usability and debuggability
Schedule: The PSF chose contractors to carry out this work in late 2019/early 2020, and commenced work in early 2020. In July 2020 the team delivered pip 20.2, which includes a beta of the new resolver. The team shipped the new resolver as default in pip 20.3, in November 2020. The work will end in December 2020/early January 2021.
Roadmap: Pip2020DonorFundedRoadmap
Code and discussion: GitHub repository for pip, Zulip livechat, and Discourse forum.
Testing: A mix of automated testing and a series of general public beta periods.
Manager: Sumana Harihareswara
Meetings and Updates: See below.
Meetings and status updates:
Meeting/update
Type
Date
Podcast interview
October 2nd, 2020
Software Developers Journey Podcast interview with Sumana Harihareswara
Podcast interview
September 29th, 2020
Podcast.__init__ episode "Dependency Management Improvements In Pip's Resolver - Episode 264"
Podcast interview
May 25th, 2020
Warehouse: OTF grant
The Packaging Working Group applied for and received a performance-based contract (like a grant) from the Open Technology Fund to implement & deploy security, localization, and accessibility improvements for Warehouse (PyPI's codebase).
Summary: See March 13 2019 blog post.
Roadmap: On Read the Docs.
Schedule: Several contractors worked, paid by PSF using the OTF funds, from March 2019 till October 2019. As of 8 October 2019, OTF-funded contractors have finished security improvements, accessibility and internationalization/localization improvements to Warehouse, and volunteers are working on Milestone 6, "Post Legacy Shutdown".
Code: GitHub repository.
Deployment: pypi.org.
Testing: WarehousePackageMaintainerTesting
Manager: Sumana Harihareswara
Meetings and Updates: See below.
Meetings and status updates from the OTF grant-funded project:
Meeting/update
Type
Date
Podcast.__init__ Episode 225: Security, UX, and Sustainability For The Python Package Index
Podcast interview
August 19th, 2019
PyPI Security and Accessibility Q1 2019 Request for Proposals period opens
Blog post
November 19th, 2018
PyPI Security and Accessibility Q1 2019 Request for Information period opens
Blog post
October 30th, 2018
Warehouse rollout
The Packaging Working Group supported the implementation & deployment of Warehouse (PyPI 2.0) to replace the legacy code base that powered legacy PyPI. Announced on PSF blog in January 2016; see its history in this April 2018 LWN article.
Summary: PSF blog post about the MOSS grant.
Roadmap: WarehouseRoadmap. As of 30 April 2018, the Warehouse team has shut down the legacy PyPI installation, and -- on a volunteer basis -- is working on Milestone 6, "Post Legacy Shutdown".
Code: GitHub repository.
Deployment: pypi.org.
Testing: See the PSF blog post about testing for the beta. (Previously: WarehousePackageMaintainerTesting, PSF blog post about testing package maintainer functionality.)
Manager: Sumana Harihareswara
Meetings and Updates: See below.
Meetings and status updates from the MOSS-funded project:
Meeting/update
Type
Date
PSF announcement of 170,000ドル MOSS award to improve sustainability of PyPI
Blog post
November 27, 2017
Developer experience audit walkthrough
in-person meeting
Tuesday, December 12, 2017
Warehouse: package manager features & question about advertising
Mailing list post
Feb. 13, 2018
PyPI & Warehouse update: redirecting & shutting down legacy by end of April
Mailing list post
March 7th, 2018
new stuff overview, beta next week, user tests, & other Warehouse updates
Mailing list post
March 14th, 2018
PyPI/Warehouse: infrastructure hardening & the CAPTCHA conundrum
Mailing list post
March 20th, 2018
PyPI/Warehouse (short) weekly report: Progress towards launch milestone
Mailing list post
April 10th, 2018
PyPI update: legacy shutdown 30 April, new classifiers page, seeking funding
Mailing list post
April 24th, 2018